Visible Camera Research Articles

Rediscovery and redescription of Oxycheila buestani Wiesner with the first female record and new methodology for observation of Oxycheila Dejean and Oxygonia Mannerheim (Coleoptera: Cicindelidae).

Biodiversity within the family Cicindelidae (Coleoptera) was studied with respect to ecology in two neighbouring localities on the western slopes of the Andes in central Ecuador. Four different methods were used along two neighbouring streams. One of them, a night vision camera, was successfully used for the first time during the night observation of adults of Oxycheila Dejean, 1825 and Oxygonia Mannerheim, 1837 to record their natural behaviour. The following six species of Cicindelidae were observed together: Oxycheila affinis W. Horn, 1900, Oxycheila pearsoni Wiesner, 1999, Oxycheila brzoskai Wiesner, 1999, Oxycheila buestani Wiesner, 1999, Pseudoxycheila chaudoiri Dokhtouroff, 1882 and Oxygonia oberthueri W. Horn, 1896. The female of the rare Oxycheila buestani is described for the first time here, and the complete redescription of this species, including colour photographs of habitus, habitat and diagnostic characters, is presented.

Design and Development of Two Wheel Drive Track Spy Robot with Night Vision Camera

Design and Development of Two Wheel Drive Track Spy Robot with Night Vision Camera

Design and GLADIS testing of a liquid tin divertor module prior to exposure in ASDEX Upgrade

Using liquid metals confined in capillary porous structures (CPSs) as a plasma-facing component (PFC) could prolong the lifetime of the divertor in the high heat flux area. However, the high atomic number of tin (Sn) limits its acceptable fraction in the main plasma. Therefore, a crucial step in developing this concept is to test it in a tokamak environment, particularly in the diverted plasma region, e.g. ASDEX Upgrade (AUG). In this paper, the design of liquid tin module (LTM) is explained, and the testing in the high heat flux device GLADIS before its use in AUG is presented. The LTM was additively manufactured using selective laser melting, consisting of a 1.5mm porous layer tungsten (W) directly attached to a solid W bulk. The LTM has a plasma-facing area of 16×40mm2 and was filled with 1.54g of Sn. In GLADIS, the module was exposed to power loads between 2 and 8MWm−2 for 1 up to 10s, first unfilled and later filled with Sn. The surface temperature was monitored with infrared imaging and pyrometry. The thermal response was used to compare with simulations in Ansys Mechanical, enabling a determination of the module’s effective thermal properties. Sn droplets could be observed on the infrared camera, until a surface temperature of about a 1000°C was reached. The enhanced wetting of tin on the plasma-facing surface, which was observed by a visible camera, suggests that there is a conditioning of the surface, possibly due to the removal of impurities and oxides. Subsequent examinations of the adjacent tile revealed minor Sn leakages emanating from the module’s edge. Furthermore, the module showed no indication of mechanical failure. Therefore, these results indicated that the LTM qualifies for the heat fluxes expected in ASDEX Upgrade.

Nighttime traffic object detection via adaptively integrating event and frame domains

Intelligent perception is crucial in Intelligent Transportation Systems (ITS), with vision cameras as critical components. However, traditional RGB cameras exhibit a significant decline in performance when capturing nighttime traffic scenes, limiting their effectiveness in supporting ITS. In contrast, event cameras possess a high dynamic range (140 dB vs. 60 dB for traditional cameras), enabling them to overcome frame degradation in low-light conditions. Recently, multimodal learning paradigms have made substantial progress in various vision tasks, such as image-text retrieval. Motivated by this progress, we propose an adaptive selection and fusion detection method that leverages both event and RGB frame domains to optimize nighttime traffic object detection jointly. To address the challenge of unbalanced multimodal data fusion, we design a learnable adaptive selection and fusion module. This module performs feature ranking and fusion in the channel dimension, allowing efficient multimodal fusion. Additionally, we construct a novel multi-level feature pyramid network based on multimodal attention fusion. This network extracts potential features to enhance robustness in detecting nighttime traffic objects. Furthermore, we curate a dataset for nighttime traffic scenarios comprising RGB frames and corresponding event streams. Through experiments, we demonstrate that our proposed method outperforms current state-of-the-art techniques in event-based, frame-based, and event and frame fusion methods. This highlights the effectiveness of integrating the event and frame domains in enhancing nighttime traffic object detection.

Observation of beamlet displacement and parallelism in NIO1

In the compact radiofrequency negative ion source NIO1 (Negative Ion Optimization phase 1), beamlet images are recorded from visible light monochrome cameras, for which several mounting positions are available. For two cameras, named CAM1 and CAM2, looking in directions perpendicular to each other and to the z axis of the beam drift tube, an extensive database of images is available, with methods and results of analysis discussed here. All NIO1 beamlets, arranged in a 3 × 3 matrix, are extracted, so CAM1 can record 3 beamlet projection groups and CAM2 also. Both cameras can be used to estimate the beam optics, depending on extracted beamlet currents, their uniformity and applied voltages; moreover CAM1 shows the filter current effects, while CAM2 images may give information on the deflection field (due to magnets inserted in the extraction grid EG and the post-acceleration grid PA). Light reflection is reduced by careful set-up, and by considering image central portion for analysis. Image luminosity can be simply fitted by Gaussian shapes, when the 3 projection groups are fairly separated. This analysis allows to estimate beamlet displacement and deflection. In general, optional algorithms for noise rejection and pre-smoothing for improving automatic recognizing of beamlet peaks is discussed, as well as the classes of available fits. Results for beamlet size, deflection and rms divergence are reported for selected datasets (within the 10 or 20 mrad goal). In some datasets, beamlet convergence was observed, indicating the need for further analysis and beam optics adjustments.

Vision ray model based stereo deflectometry for the measurement of the specular surface

As a slope measuring specular surface measurement technique, deflectometry relies on the characterization of camera vision ray, which is commonly achieved with the distortion compensated pinhole model. However, this model is an approximate and inadequate description of lens layout and distortion, leading to low reproducibility of calibration and uncertainty in vision ray, and hence surface uncertainty. We implement the vision ray model (VRM) to stereo deflectometry, providing a pixelwise characterization of the vision ray that is free of intrinsic lens layout and distortion representation. Based on the VRM, the calibration method and surface reconstruction algorithm are proposed. We demonstrate with experimental study that the VRM can improve both the calibration performance and the surface measurement accuracy. The experimental results of a flat optical element and a concave spherical mirror indicate that the VRM based stereo deflectometry can achieve ten nanometers level RMS error.

In situ spin coater for multimodal grazing incidence x-ray scattering studies.

We present herein a custom-made, in situ, multimodal spin coater system with an integrated heating stage that can be programmed with spinning and heating recipes and that is coupled with synchrotron-based, grazing-incidence wide- and small-angle x-ray scattering. The spin coating system features an adaptable experimental chamber, with the ability to house multiple ancillary probes such as photoluminescence and visible optical cameras, to allow for true multimodal characterization and correlated data analysis. This system enables monitoring of structural evolutions such as perovskite crystallization and polymer self-assembly across a broad length scale (2 Å-150nm) with millisecond temporal resolution throughout a complete thin film fabrication process. The use of this spin coating system allows scientists to gain a deeper understanding of temporal processes of a material system, to develop ideal conditions for thin film manufacturing.

Acoustic sources identified using a microphone phased array during a rocket engine test

A new phased array of microphones, suitable for the harsh environment of a rocket launch, was built and tested during a static firing of an RS-25 engine. It uses 70 piezo-resistive, dynamic pressure sensors, optimally distributed on a 10.5-ft diameter open frame dome structure and has a 200-ft long cable bundle to carry the signals to a weather-protected cabinet containing the data systems. The test stand was imaged using an infra-red camera and a visible wavelength camera, and the beamformed noise maps were superimposed on the photographs. The first-time data from a full-scale engine test stand showed that the plume deflector at the bottom of the engine was the primary noise source. The openings of the test stand around the nozzle exit were also found to be noise sources particularly at higher frequencies. The final goal is to utilize the array during NASA’s Artemis-II launch at Kennedy Space Center.

THPoseLite, a Lightweight Neural Network for Detecting Pose in Thermal Images

Nowadays, Smart Environments (SEs) enable the monitoring of people with physical disabilities by incorporating activity recognition. Thermal cameras are being incorporated as they preserve privacy. Some Deep Learning (DL) solutions use the pose of the users because it removes external noise. Although there are robust DL solutions in the visible spectrum, they fail in the thermal domain. Thus, we propose THPoseLite (Thermal Human Pose Lite), a Convolutional Neural Network (CNN) based on MobileNetV2 that extracts pose from Thermal Images (TIs). In a novel way, an auto-labeling approach has been developed. It includes a background removal using an optical flow estimator. It also integrates Blazepose (a pose estimator for Visible spectrum images (VSIs)) to obtain the poses in the pre-processed TIs. Results show that the pre-processing increases the percentage of detected poses by Blazepose from 19.55% to 76.85%. This allows the recording of Human Pose Estimation (HPE) datasets in the visible spectrum without requiring visible spectrum cameras or manually annotating datasets. Furthermore, THPoseLite has been embedded in an Internet of Things (IoT) device incorporating an edge Tensor Processing Unit (TPU) accelerator, which can process TIs recorded at 9 Frames Per Second (FPS) in real-time (12.28 FPS). It requires fewer than 6W of energy to run. It has been achieved using model quantization, decreasing the accuracy in estimating the poses by only 1%. The MSE of MobileNetV2 in test images is 35.48, obtaining accurate poses in 21% of the images that Blazepose is not able to detect any pose.

Automated IBEX crawler for PAUT inspection for in-service ferromagnetic assets

In order to guarantee the safety of the gas production plants, Air Liquide conducts NDT inspections regularly closely following the assets’ maintenance plans. One high criticality asset is the Pressure Swing Adsorber (PSA), found in the last stages of the hydrogen production process, where product purification takes place. These adsorbers are pressure vessels and need to be inspected for hydrogen enhanced fatigue cracks. Conventional inspection would require stopping site operation, building scaffolding for the totality of the height, which depending on design can be up to 15 meters tall and 6 meters in diameter, grinding of the inspection surface to remove the paint, conducting the inspection manually by 2 operators, and repainting once inspections are concluded. Commonly, inspection plans for these high-criticality assets outline a high inspection frequency, resulting in elevated costs associated with logistics, personnel and time and monetary resources required to plan and execute each inspection. This necessity is what drove Air Liquide R&D and INTACT to create a partnership and develop an automated crawler that can be deployed on in-service ferromagnetic assets for Phased Array Ultrasonic Testing (PAUT) inspection, using Total Focusing Method (TFM) with INTACT proprietary imaging and software. The robot, named IBEX Crawler, is deployed at the lowest part of the vessel and controlled safely at the ground level by a single operator. It is able to change directions to accommodate the inspection of longitudinal and circular welds. It includes an encoding system that enables operators to exactly locate any indications present. The crawler’s payload includes an electronic 128 elements PAUT board controlled by fiber optics, laser tracking and machine vision camera. The inspection results are analyzed by 3D reconstruction and managed by mechanical integrity assessments proprietary technology, calculating the stability of the found indications and using fracture mechanics and Fitness-For-Service (FFS) to estimate the residual lifetime of the indications and in consequence, the interval between inspections to develop a more accurate maintenance plan. Today INTACT’s IBEX crawlers travel worldwide between Air Liquide sites to ensure the integrity of the assets.

Porcospino, spined single-track mobile robot for inspection of narrow spaces

AbstractThis paper discusses the design and the experimental tests on Porcospino, a bio-inspired single-track mobile robot for inspection of unstructured environments characterized by narrow spaces. It is an evolution of SnakeTrack, a single-track robot with steering capabilities; differently from SnakeTrack, the track modules of Porcospino are characterized by elastic spines, which improve traction on uneven and irregular terrains. The main body is a vertebral column, comprising a series of vertebrae connected by compliant joints and two end modules. Each end module carries two sprockets, sharing a common actuator, to drive the single peripherical track. Moreover, each end module hosts an actuator for track steering. The remaining mobilities of the vertebral column allow it to cope passively with the terrain profile, to enhance traction. The control unit, batteries, drivers, and environmental sensors are placed along the vertebral column. Both the end modules are equipped with a camera for intermittent vision, which is possible thanks to openings realized on the track modules. The experimental campaign on the first Porcospino prototype is discussed, highlighting the differences with its earlier version.

Acoustic Vehicle Classification Using Mel-Frequency Features with Long Short-Term Memory Neural Networks

Monitoring vehicle traffic at a large scale is a challenging task for authorities, particularly considering the high cost of traffic sensors such as vision cameras. To meet the growing demand for more accurate traffic monitoring, the use of traffic sounds has become a popular approach, as it provides insight into the types of traffic present. This paper reports on an approach to vehicle classification based on acoustic signals, using the Mel-Frequency Cepstral Coefficients (MFCC) and the Long Short-Term Memory (LSTM) networks. This study exhibited classification accuracy scores of 82-86.2% across four vehicle categories: motorcycle, car, truck, and no traffic. The results demonstrated that large-scale, low-cost acoustic processing can be effectively used for vehicle monitoring.

Multi-timescale boosting for efficient and improved event camera face pose alignment

The success of event camera (EC) vision in certain types of applications has been steadily shown thanks to energy-efficient sparse sensing, high dynamic range, and extremely high temporal resolution. However, the utilization of ECs for facial processing tasks has remained rather limited. To enable high energy efficiency for large face pose alignment, which is a crucial facial pre-processing stage, we aim at leveraging EC by effective adaptation of the processing rate proportional to facial movement intensity. For this purpose, we propose a novel alternative to the commonly employed constant time frame and event count frame strategies which combines their advantages and provides the benefits of supervised learning. This is realized by a multi-timescale boosting framework that can generate highly sparse pose-events at a variable rate via detection-based online timescale selection. Although detectors of multiple scales with boosted sensitivities operate as a cascade, our method provides minimal delay essential for real-time applications. Comprehensive evaluations show that the proposed multi-timescale processing substantially improves the performance–efficiency trade-off over single-timescale frames and markedly over event count frames. Mega-floating-point-operations-per-second ranges from 2.5 at the moderate motion clips to 6.5 at the intense motion clips, with negligible computation in the absence of activity. Also, alignment errors are considerably reduced by online selection of small timescales at fast head motion and of bigger timescales at slower motion or local activity of lips and eyes. Being orthogonal and complementary to spatial domain techniques, the proposed approach can also be conveniently integrated with future advances for further performance/efficiency improvements or for alignment extensions.

Effect of wind turbulences on the burning of a rockrose hedge

With climate change, complex fire scenarios at Wildland Urban Interfaces (WUI) have become a growing concern for authorities worldwide. Such scenarios involve unfavorable wind conditions, which greatly increase the fire spread near communities and makes the intervention of firefighters even more complex. To design effective fire safety regulations to protect communities, it is important to have a clear understanding of the role of wind in the burning of vegetation to help identify the vectors of fire spread within the defensible zone. In this context, this paper attempts to relate the properties of the flames generated by the burning of a reconstructed rockrose hedge to the atmospheric wind turbulence using four field-scale experiments. The experimental setup includes 3 visible cameras to record the flame geometry, extracted by post-processing; 3 pairs of radiant and total heat fluxes to record the heat fluxes downstream of the hedge, and an anemometer to measure the wind speed and direction. The results show that the wind at the experimental site is composed of wall-bounded turbulent structures coupled with strong and short intermittent events such as gusts. Regarding the fire, the four replicates show similarities in phase durations and dynamics, but revealed dependencies on Reynolds number and turbulent kinetic energy when averaging over the fully developed flame phase. In addition, a slight flapping motion of the ensemble flame was observed and associated with wave numbers of the order of the turbulent cascade in the kinetic energy power spectra. Finally, the analysis of wind and fire intermittencies revealed that strong and short wind events generate large fluctuations of flame geometry and received heat fluxes that increase with the characteristic time of the event considered.

Visual camera relocalization using both hand-crafted and learned features

The localization of the camera is essential in AR, MR, and robotics. Diverse pipelines employ a hand-crafted or learning based way to predict the camera pose as per the task. In the localization process, both weaknesses and strengths are maintained. However, few current frameworks consider these two features simultaneously. In this study, a novel relocalization pipeline for RGB or RGB-D input is proposed, including a coarse stage with learned features, further refinement with hand-crafted features, and a stable process to measure the confidence of both stages for improving localization robustness. Instead of directly regressing the camera pose, the coarse procedure uses registration to the known source and predicted weighted target point cloud to obtain the initial result. Therefore, we design a deep network called PGNet to construct the weighted target point cloud with the image and previous poses as inputs. Moreover, in consideration of dynamic surroundings, we add a segmentation branch distinguishing each point as either fixed or dynamic with the purpose of promoting dynamic perception. Correspondingly, the segmentation-extended Chamfer Distance is added to optimize PGNet. During the pose refinement, the feature space is established via hand-crafted feature extraction and matching on the training set. Based on the coarse pose, we obtain the accurate pose by applying Kabsch or Perspective-n-Point (PnP) algorithm to point-to-point correspondences built through searching the space and matching Oriented Fast and Rotated Brief (ORB) features. Furthermore, an additional process is presented by defining coarse and refinement metrics to gain a more stable performance. Finally, experiments on both static and dynamic scenes are conducted. On the one side, the results demonstrate the state-of-the-art performance over other existing methods on 7 Scenes, INDOOR-6, Cambridge Landmarks and TUM RGB-D. On the other side, the positive effects of the pose learning part, dynamic branch, confidence regression and hand-crafted feature based refinement are also provided.

Design and Development of an Ultraviolet All-Sky Imaging System.

All-sky cameras capture a panoramic view of the full sky from horizon to horizon to generate a wide-angle image of the observable sky. State-of-the-art all-sky imagers are limited to imaging in the visible and infrared spectrum and cannot image in the UV spectrum. This article describes the development of an all-sky imaging system capable of capturing 130° wide-angle sky images from horizon to horizon in the UV-AB spectrum. The design of the UV all-sky imaging system is based on low-cost, accessible, and scalable components to develop multiple images that can be deployed over a wider geographical area. The spectral response of the camera system has been validated in the UV spectrum (280-420 nm) using a monochromatic UV beam with an average power output of 22 nW. UV all-sky imaging systems complement existing infrared and visible all-sky cameras. They have wide applications in astronomy, meteorology, atmospheric science, vulcanology, meteors and auroral monitoring, and the defence sector.

Feasibility of the EDICAM camera for runaway electron detection in JT-60SA disruptions

The visible camera system EDICAM (Event Detection Intelligent Camera), recently installed on JT-60SA, is simulated to assess whether it can be used for measuring synchrotron radiation from relativistic runaway electrons. In this simulation, the SOFT synthetic synchrotron diagnostic framework is used to compute the synthetic synchrotron camera images from a JT-60SA-like disruption modelled with the DREAM disruption simulation code. In the studied scenario, a large amount of argon is added to the plasma, and a disruption is simulated by starting a prescribed exponential temperature drop and finishing with further cooling provided by the argon in a self-consistent simulation of the current quench. The background plasma evolution is calculated by DREAM self-consistently with the fast electron population, which is modelled kinetically. The resulting runaway electron distribution function along with the parameters of the EDICAM visible camera system are used as an input to the SOFT framework to assess the feasibility of the camera for runaway electron detection. We find that the runaway electron beam formed in the disruption can produce synchrotron radiation observable by the EDICAM system, thus enabling the use of the EDICAM for the characterization of runaway electron beams.

An integrated and real-time social distancing, mask detection, and facial temperature video measurement system for pandemic monitoring

This paper presents a new Edge-AI algorithm for real-time and multi-feature (social distancing, mask detection, and facial temperature) measurement to minimize the spread of COVID-19 among individuals. COVID-19 has extenuated the need for an intelligent surveillance video system that can monitor the status of social distancing, mask detection, and measure the temperature of faces simultaneously using deep learning (DL) models. In this research, we utilized the fusion of three different YOLOv4-tiny object detectors for each task of the integrated system. This DL model is used for object detection and targeted for real-time applications. The proposed models have been trained for different data sets, which include people detection, mask detection, and facial detection for measuring the temperature, and evaluated on these existing data sets. Thermal and visible cameras have been used for the proposed approach. The thermal camera is used for social distancing and facial temperature measurement, while a visible camera is used for mask detection. The proposed method has been executed on NVIDIA platforms to assess algorithmic performance. For evaluation of the trained models, accuracy, recall, and precision have been measured. We obtained promising results for real-time detection for human recognition. Different couples of thermal and visible cameras and different NVIDIA edge platforms have been adopted to explore solutions with different trade-offs between cost and performance. The multi-feature algorithm is designed to monitor the individuals continuously in the targeted environments, thus reducing the impact of COVID-19 spread.

Applying UAS Technologies at Night during a Wildlife Hazard Assessment

Airports operating under the Code of Federal Regulations Part 139 should conduct a wildlife hazard assessment (WHA) when some wildlife-strike events have occurred at or near the airport. The WHA must be conducted by a Qualified Airport Wildlife Biologist (QAWB). The required elements in a WHA include the identification of the wildlife species observed and their numbers and the location of features on and near the airport that could attract wildlife. The collection of data pertaining to mammal populations during a WHA is generally time-consuming, labor-intensive, and costly. Protocols to collect this data include trapping and marking animals and systematic surveys using a spotlight and/or night vision equipment. The purpose of this study was to investigate how UAS technologies could be effectively applied to streamline the QAWB efforts during a WHA at night. Researchers, including students, used a DJI Mavic 2 Enterprise Dual with visual and thermal cameras and a spotlight, as well as a Matrice 210 with a Zenmuse XT2 thermal camera to collect data. Data were collected in a farmland area located two nautical miles south of Daytona Beach International Airport. We applied multiple strategies to mitigate the risks associated with drone operations in an airport environment. The safe application of UAS to streamline the WHA process is anticipated to provide several benefits to the airport operator, including task completion in reduced time, enhanced level of accuracy during the data collection process, reduced risks for the QAWB, and cost efficiencies. Most importantly, researchers expect to develop benchmark safety protocols that can facilitate the effective integration of UAS into the airport environment.

Traffic Noise Assessment Using Intelligent Acoustic Sensors (Traffic Ear) and Vehicle Telematics Data.

Here, we introduce Traffic Ear, an acoustic sensor pack that determines the engine noise of each passing vehicle without interrupting traffic flow. The device consists of an array of microphones combined with a computer vision camera. The class and speed of passing vehicles were estimated using sound wave analysis, image processing, and machine learning algorithms. We compared the traffic composition estimated with the Traffic Ear sensor with that recorded using an automatic number plate recognition (ANPR) camera and found a high level of agreement between the two approaches for determining the vehicle type and fuel, with uncertainties of 1-4%. We also developed a new bottom-up assessment approach that used the noise analysis provided by the Traffic Ear sensor along with the extensively detailed urban mobility maps that were produced using the geospatial and temporal mapping of urban mobility (GeoSTMUM) approach. It was applied to vehicles travelling on roads in the West Midlands region of the UK. The results showed that the reduction in traffic engine noise over the whole of the study road was over 8% during rush hours, while the weekday-weekend effect had a deterioration effect of almost half. Traffic noise factors (dB/m) on a per-vehicle basis were almost always higher on motorways compared the other roads studied.