Fast Frame Research Articles

High‐Performance Infrared Scene Generation Using a Double‐S Microbridge Photothermal Film Emitter With Ultralarge Array

AbstractInfrared emitters are highly desirable for applications in infrared imaging and stealth technology. Photothermal film emitters, which are driven by light, have garnered considerable interest due to their excellent photothermal properties and ease of fabrication. However, existing photothermal films often suffer from low photothermal conversion efficiency (PCE) and limited array scale. Here, a photothermal film emitter featuring a double‐S microbridge structure is presented, which comprehensively considering the PCE and frame rate. Initially, Si microbridges are fabricated within each microstructure to support the photothermal film. The pattern structure is then arranged into a double‐S configuration to manipulate the thermal conductivity. By optimizing the length and width of the double‐S pattern, high PCE and fast frame rates are achieved. The double‐S microbridge structure photothermal film emitter is fabricated with a 4‐inch diameter, forming an ultra‐large array with over 3000 × 3000 pixels. The absorption and photothermal conversion characteristics are studied, revealing a maximum temperature of 582.70 K and a highest PCE of 16.32%. Time‐dependent photothermal response demonstrated the emitter's rapid heat storage and release capabilities. Additionally, an infrared scene generator is realized. These results position the photothermal film emitter as a promising solution for infrared imaging and dynamic scene generation.

Deep learning-driven optimization of real-time ray tracing for enhanced immersive virtual reality

Photorealistic rendering is essential for immersive virtual reality and real-time ray tracing is required for this, yet the necessary computational load has so far hindered its use in scenarios where a fast response and high frame rates are paramount. To overcome this challenge, we present a deep learning-based model that optimises the computational load of ray tracing through utilising convolutional neural networks (CNNs) to predict lightscene interactions. We achieve this by training CNNs from datasets with offline ray-traced images. Using this learning process, we approximate the contents of light transport simulations from scene point sampling. The use of this optimised algorithm in virtual reality scenes, thus, enables renderings of more complex scenes, with dynamic lighting, complex geometry and interactive elements under high frame rates, keeping users immersed and responsive to events. Our results show that this method achieves visual similarities to traditional ray tracing, which is photorealistic rendering, but can be performed in real-time in virtual reality. We anticipate this work leads to many potential applications of ray tracing in many scenarios in virtual reality, such as gaming, architectural rendering and training.

Unsupervised learning-enabled pulsed infrared thermographic microscopy of subsurface defects in stainless steel

Metallic structures produced with laser powder bed fusion (LPBF) additive manufacturing method (AM) frequently contain microscopic porosity defects, with typical approximate size distribution from one to 100 microns. Presence of such defects could lead to premature failure of the structure. In principle, structural integrity assessment of LPBF metals can be accomplished with nondestructive evaluation (NDE). Pulsed infrared thermography (PIT) is a non-contact, one-sided NDE method that allows for imaging of internal defects in arbitrary size and shape metallic structures using heat transfer. PIT imaging is performed using compact instrumentation consisting of a flash lamp for deposition of a heat pulse, and a fast frame infrared (IR) camera for measuring surface temperature transients. However, limitations of imaging resolution with PIT include blurring due to heat diffusion, sensitivity limit of the IR camera. We demonstrate enhancement of PIT imaging capability with unsupervised learning (UL), which enables PIT microscopy of subsurface defects in high strength corrosion resistant stainless steel 316 alloy. PIT images were processed with UL spatial–temporal separation-based clustering segmentation (STSCS) algorithm, refined by morphology image processing methods to enhance visibility of defects. The STSCS algorithm starts with wavelet decomposition to spatially de-noise thermograms, followed by UL principal component analysis (PCA), fine-tuning optimization, and neural learning-based independent component analysis (ICA) algorithms to temporally compress de-noised thermograms. The compressed thermograms were further processed with UL-based graph thresholding K-means clustering algorithm for defects segmentation. The STSCS algorithm also includes online learning feature for efficient re-training of the model with new data. For this study, metallic specimens with calibrated microscopic flat bottom hole defects, with diameters in the range from 203 to 76 µm, were produced using electro discharge machining (EDM) drilling. While the raw thermograms do not show any material defects, using STSCS algorithm to process PIT images reveals defects as small as 101 µm in diameter. To the best of our knowledge, this is the smallest reported size of a sub-surface defect in a metal imaged with PIT, which demonstrates the PIT capability of detecting defects in the size range relevant to quality control requirements of LPBF-printed high-strength metals.

Single-photon counting detectors for diffraction-limited light sources

When first introduced, single-photon counting detectors reshaped crystallography at synchrotrons. Their fast readout speed enabled, for example, shutter-less data collection and fine slicing of the rotation angle and boosted the development of new experimental techniques like ptychography. Under optimal conditions, single-photon counting detectors provide an unlimited dynamic range with image noise only limited by the Poisson statistics of the incoming photons. Counting the pulses from individual photons, essentially what made the detectors so successful, also causes the main drawback, which is the loss of efficiency at high photon fluxes due to pulse pileup in the analog front end. To fully take advantage of diffraction-limited light sources, the next-generation single-photon counters need to improve their count rate capabilities in the same order of magnitude as the increased flux. Moreover, fast frame rates (a few kHz) are required to cope with the shorter dwell time achievable, thanks to the higher flux. Detector architecture with multiple comparators and counters can open new possibilities for energy-resolved imaging, while interpixel communication can overcome the issues arising from charge sharing and reduce the loss of efficiency at the pixel corners. Coupling single-photon counting detectors to high-Z sensors for hard X-ray detection (>20 keV) and to low-gain avalanche diodes (LGADs) for soft X-rays is also necessary to make use of the increased coherence of the new light sources over the full radiation spectrum. In this paper, we present possible strategies to improve the performance of single-photon counting detectors at the fourth-generation synchrotron sources and compare them to charge integrating detectors.

TomoCAM: fast model-based iterative reconstruction via GPU acceleration and non-uniform fast Fourier transforms.

X-ray-based computed tomography is a well established technique for determining the three-dimensional structure of an object from its two-dimensional projections. In the past few decades, there have been significant advancements in the brightness and detector technology of tomography instruments at synchrotron sources. These advancements have led to the emergence of new observations and discoveries, with improved capabilities such as faster frame rates, larger fields of view, higher resolution and higher dimensionality. These advancements have enabled the material science community to expand the scope of tomographic measurements towards increasingly in situ and in operando measurements. In these new experiments, samples can be rapidly evolving, have complex geometries and restrictions on the field of view, limiting the number of projections that can be collected. In such cases, standard filtered back-projection often results in poor quality reconstructions. Iterative reconstruction algorithms, such as model-based iterative reconstructions (MBIR), have demonstrated considerable success in producing high-quality reconstructions under such restrictions, but typically require high-performance computing resources with hundreds of compute nodes to solve the problem in a reasonable time. Here, tomoCAM, is introduced, a new GPU-accelerated implementation of model-based iterative reconstruction that leverages non-uniform fast Fourier transforms to efficiently compute Radon and back-projection operators and asynchronous memory transfers to maximize the throughput to the GPU memory. The resulting code is significantly faster than traditional MBIR codes and delivers the reconstructive improvement offered by MBIR with affordable computing time and resources. tomoCAM has a Python front-end, allowing access from Jupyter-based frameworks, providing straightforward integration into existing workflows at synchrotron facilities.

Interação de Mercados: Agenciamento de Interdefinição de Mercado de Alimentos

ABSTRACT Objective: in order to collaborate with the ongoing research program on how markets intersect and overlap, this theoretical essay introduces the notion of market interdefinition agencing. Theoretical framework: the theoretical framework draws on previous constructivist market studies on the process of framing, overflowing, translation, and agencing in a market interaction context. Methods: to illustrate the market interdefinition agencing, this article uses a theoretical essay on slow and fast food markets. Results: this essay depicts how slow food can be seen as an overflow in opposition to fast food framing, agencing opposite sociotechnical elements for the construction of this new frame, and how slow food also contributes to a new fast food frame. Conclusion: first, this essay shows that both food markets entangle and disentangle sociotechnical elements that move away from or closer to each other, translating and agencing their sociotechnical arrangements. Second, market interdefinition agencing is delineated as a process of market interrelation that happens through the capacity of actors to shape markets, where a previous market is used as a reference to influence another market frame.

Cascaded adaptive global localisation network for steel defect detection

Defect detection is crucial in ensuring the quality of steel products. This paper proposes a novel deep neural network, cascaded adaptive global location network (CAGLNet), for detecting steel surface defects. The main objective of this study is to address the challenges associated with the irregular shape and dense spatial distribution of defects on steel. To achieve this goal, CAGLNet integrates a feature extraction network that combines residual and feature pyramid networks, a cascade adaptive tree-structure region proposal network (CAT-RPN) that eliminates the need for prior knowledge, and a global localisation regression for steel defect detection. This paper evaluates the effectiveness of CAGLNet on the NEU-DET dataset and demonstrates that the proposed model achieves an average accuracy of 85.40% with a fast frames per second of 10.06, outperforming those state-of-the-art methods. These results suggest that CAGLNet has the potential to significantly improve the effectiveness of defect detection in industrial production processes, leading to increased production yield and cost savings. Abbreviations: AT-RPN, adaptive tree-structure region proposal network; CAGLNet, cascaded adaptive global location network; CAT-RPN, cascade adaptive tree-structure region proposal network; CNN, convolutional neural network; DNN, deep neural network; EPNet, edge proposal network; FPN, feature pyramid network; FCOS, fully convolutional one-stage detector; FPS, frames per second; GMM, Gaussian mixture model; IoU, intersection-over-union; ROIAlign, region of interest align; RPN, region proposal network; ResNet, residual network; ResNet50_FPN, residual network and feature pyramid network; SABL, side aware boundary localisation; SSD, single-shot multiBox detector; TPE, Tree-structured Parzen estimator

A Semiautonomous Control Strategy Based on Computer Vision for a Hand–Wrist Prosthesis

Alleviating the burden on amputees in terms of high-level control of their prosthetic devices is an open research challenge. EMG-based intention detection presents some limitations due to movement artifacts, fatigue, and stability. The integration of exteroceptive sensing can provide a valuable solution to overcome such limitations. In this paper, a novel semiautonomous control system (SCS) for wrist–hand prostheses using a computer vision system (CVS) is proposed and validated. The SCS integrates object detection, grasp selection, and wrist orientation estimation algorithms. By combining CVS with a simulated EMG-based intention detection module, the SCS guarantees reliable prosthesis control. Results show high accuracy in grasping and object classification (≥97%) at a fast frame analysis frequency (2.07 FPS). The SCS achieves an average angular estimation error ≤18° and stability ≤0.8° for the proposed application. Operative tests demonstrate the capabilities of the proposed approach to handle complex real-world scenarios and pave the way for future implementation on a real prosthetic device.

Growth Anisotropy and Morphology Evolution of Line Defects in Monolayer MoS2 : Atomic-Level Observation, Large-Scale Statistics, and Mechanism Understanding.

Understanding the growth behavior and morphology evolution of defects in 2D transition metal dichalcogenides is significant for the performance tuning of nanoelectronic devices. Here, the low-voltage aberration-corrected transmission electron microscopy with an in situ heating holder and a fast frame rate camera to investigate the sulfur vacancy lines in monolayer MoS2 is applied. Vacancy concentration-dependent growth anisotropy is discovered, displaying first lengthening and then broadening of line defects as the vacancy densifies. With the temperature increase from 20 °C to 800 °C, the defect morphology evolves from a dense triangular network to an ultralong linear structure due to the temperature-sensitive vacancy migration process. Atomistic dynamics of line defect reconstruction on the millisecond time scale are also captured. Density functional theory calculations, Monte Carlo simulation, and configurational force analysis are implemented to understand the growth and reconstruction mechanisms at relevant time and length scales. Throughout the work, high-resolution imaging is closely combined with quantitative analysis of images involving thousands of atoms so that the atomic-level structure and the large-area statistical rules are obtained simultaneously. The work provides new ideas for balancing the accuracy and universality of discoveries in the TEM study and will be helpful to the controlled sculpture of nanomaterials.

FLAVR: flow-free architecture for fast video frame interpolation

FLAVR: flow-free architecture for fast video frame interpolation

Fast and accurate semantic segmentation of road crack video in a complex dynamic environment

ABSTRACT This paper proposes a fast and accurate semantic segmentation of road crack video in a complex dynamic environment. First, a fast key frame selection algorithm is designed by combining the interframe dissimilarity constrained video frame difference method (FDM) and shot interval sampling method (SISM). Second, the complex and dynamic characteristics of application scenarios are studied, and a high-precision crack semantic segmentation DBPNet containing a densely diverse branch module (DDBM) and diverse branch pyramid module (DBPM) is proposed, which can not only focus on the local aggregation of the same scale feature but also fuse and reconstruct long-distance related semantic features. Finally, a dynamic video data set of cracks in complex environments, including multiple types of interference, different weather and lighting is established, and experimental analysis is carried out. The results show that the proposed method can improve the efficiency of video crack detection by 5–7 times compared to the method without using key frames, and the detection accuracy can reach 72.85%, which can adapt to dynamic video changes in a variety of complex scenes. The proposed network, DBPNet, outperforms the current state-of-the-art methods in road crack semantic segmentation on challenging data sets in terms of both Dice and MIOU.

Interlacing in Atomic Resolution Scanning Transmission Electron Microscopy.

Fast frame rates are desirable in scanning transmission electron microscopy for a number of reasons: controlling electron beam dose, capturing in situ events, or reducing the appearance of scan distortions. While several strategies exist for increasing frame rates, many impact image quality or require investment in advanced scan hardware. Here, we present an interlaced imaging approach to achieve minimal loss of image quality with faster frame rates that can be implemented on many existing scan controllers. We further demonstrate that our interlacing approach provides the best possible strain precision for a given electron dose compared with other contemporary approaches.

Multi-scale time-resolved electron diffraction: A case study in moiré materials

Ultrafast-optical-pump — structural-probe measurements, including ultrafast electron and x-ray scattering, provide direct experimental access to the fundamental timescales of atomic motion, and are thus foundational techniques for studying matter out of equilibrium. High-performance detectors are needed in scattering experiments to obtain maximum scientific value from every probe particle. We deploy a hybrid pixel array direct electron detector to perform ultrafast electron diffraction experiments on a WSe2/MoSe2 2D heterobilayer, resolving the weak features of diffuse scattering and moiré superlattice structure without saturating the zero order peak. Enabled by the detector’s high frame rate, we show that a chopping technique provides diffraction difference images with signal-to-noise at the shot noise limit. Finally, we demonstrate that a fast detector frame rate coupled with a high repetition rate probe can provide continuous time resolution from femtoseconds to seconds, enabling us to perform a scanning ultrafast electron diffraction experiment that maps thermal transport in WSe2/MoSe2 and resolves distinct diffusion mechanisms in space and time.

Visible and short-wave infrared fiber-based snapshot imaging spectrometer with a custom high-throughput relay system

This paper presents the design and fabrication of a fiber-based snapshot imaging spectrometer working in both visible (490 nm-732 nm) and short-wave infrared (1090 nm - 1310 nm) ranges. To maximize the light collection efficiency, a custom relay system with 0.25 NA and 20 mm field of view (FOV) was designed and integrated. The bench setup showed that the custom relay system could fully resolve 10 µm fiber cores over the entire FOV among visible and short-wave infrared ranges. The numerical aperture (NA) match provided a 2.07X fold throughout improvement in the visible range and about 10X fold in the SWIR range compared to the previous generations, enabling imaging with a fast frame rate and under low illumination conditions. The presented imaging spectrometer generated spectral datacubes with 35000 spatial samplings and 23 spectral channels. Spectral urban imaging results obtained by the spectrometer in both visible and SWIR ranges are presented. Finally, we collected spectral images of apple bruising to show potential applications in the food quality industry.

High-speed and wide-field nanoscale table-top ptychographic EUV imaging and beam characterization with a sCMOS detector.

We present high-speed and wide-field EUV ptychography at 13.5 nm wavelength using a table-top high-order harmonic source. Compared to previous measurements, the total measurement time is significantly reduced by up to a factor of five by employing a scientific complementary metal oxide semiconductor (sCMOS) detector that is combined with an optimized multilayer mirror configuration. The fast frame rate of the sCMOS detector enables wide-field imaging with a field of view of 100 µm × 100 µm with an imaging speed of 4.6 Mpix/h. Furthermore, fast EUV wavefront characterization is employed using a combination of the sCMOS detector with orthogonal probe relaxation.

From STEM to 4D STEM: Ultrafast Diffraction Mapping with a Hybrid-Pixel Detector

Abstract4D scanning transmission electron microscope (STEM) techniques have been increasingly featured among the electron microscopy characterization approaches, as they provide a perspective of improved information retrieval from samples overall. To make 4D STEM experiments as viable as conventional STEM image acquisition, the recording of diffraction patterns with a pixelated detector at fast frame rates, sufficient sensitivity to capture single electron hits, and high dynamic range is necessary. This paper addresses the recent development in hybrid-pixel detector technology that now allows 4D STEM experiments with a similar setup to conventional STEM imaging with pixel collection time under 10 µs. Application examples on virtual STEM detectors and crystal phase-orientation mapping are presented.

Second harmonic generation laser scanning microscopy with intrinsic first-order modulation

The contrast or signal-to-noise ratio (SNR) in images is a crucial parameter that determines the quality of images in second harmonic generation laser scanning microscopy (SHG-LSM). With a better image contrast/SNR, SHG-LSM can be performed at a faster frame rate or deeper tissue locations without compromising image quality. In this work, we present a novel strategy based on the SHG signals of first-order modulation (1 M) to enhance the image contrast/SNR. Notably, a photodetector (i.e., photomultiplier tube) has a better signal SNR during the photon-to-electron conversion process at the frequency of 1 M, which is equivalent to the laser pulse repetition frequency. The improved signal SNR thus enhances the image quality, which agrees well with the results of the measured electrical spectra. A remarkable increment of image contrast by more than a factor of two has been obtained in images by using 1 M. In addition, the image acquisition time is also shortened by a factor of 2.5 compared to that for acquiring images with similar contrast taken without signal modulation. Furthermore, by analyzing the dependencies of image contrast/SNR on sample depth, we demonstrate that the images obtained with 1 M have better quality than those obtained without signal modulation at the same imaging depth.

Evaluating direct detection detectors for short-range order characterization of amorphous materials by electron scattering

The introduction of direct electron detectors (DEDs) to transmission electron microscopy has set off the ‘resolution revolution’, especially for cryoTEM low-dose imaging of soft matter.In comparison to traditional indirect electron detectors such as Charged-Coupled Devices (CCD), DEDs show an improved modulation transfer function (MTF) and detective quantum efficiency (DQE) across all spatial frequencies, as well as faster frame rates which enable single electron counting.The benefits of such characteristics for imaging, spectroscopy and electron holography have been demonstrated previously. However, studies are lacking on the application of DEDs for localized characterization of short- to medium- range-order (SRO, MRO) in amorphous materials using electron scattering.Therefore, we evaluate the performance of a Monolithic Active Pixel Sensor DED for the characterization of SRO and MRO in nanoscale volumes of amorphous materials, using SiO2 and Ta2O5 thin films as test cases. The performance of the detector is compared systematically to electron scattering measurements recorded on an indirect detector (CCD) using 200 keV electrons and electron doses starting at approximately 500e−Å2 . In addition, the effects of sample cooling and energy-filtering on the measured SRO of the oxides were investigated.We demonstrate that the performance of the DED resulted in improved SRO characterization in comparison to that obtained from the CCD measurements. The DED enabled to achieve a larger measured maximal scattering vector, ∼16.51Å compared to ∼151Å, for the CCD. Furthermore, an improved signal-to-noise ratio of approximately two-fold was observed across all spatial frequencies for both 200 keV and 80 keV electrons. These improvements are shown to result from the superior DQE of the DED. Consequently, the DED measurements enabled to determine the coordination numbers of atomic bonds more accurately.We expect that further benefits of the DED for S/MRO characterization will be highlighted for ultra- sensitive materials that cannot withstand electron doses above several e−Å2 .

Single-Shot Reconfigurable Femtosecond Imaging of Ultrafast Optical Dynamics.

Understanding ultrafast dynamics in the femtosecond timescale plays a pivotal role in fundamental research and technology innovation. Spatiotemporal observation of those events in real-time requires imaging speeds greater than 1012 frames per second (fps), far beyond the fundamental speed limits of the ubiquitous semiconductor sensor technologies. In addition, a majority of femtosecond events are non-repeatable or difficult-to-repeat since they either work in a highly unstable nonlinear regime or require extreme or rare conditions to initiate. Therefore, the traditional pump-probe imaging approach fails since it heavily depends on precise event repetition. Single-shot ultrafast imaging emerges as the only solution; however, existing techniques cannot reach more than 15×1012 fps, and they only record an insufficient number of frames. Compressed ultrafast spectral photography (CUSP) is proposed to overcome these limitations. Here, CUSP's full design space is explored by manipulating the ultrashort optical pulse in the active illumination. Via parameter optimization, an extraordinarily fast frame rate of 219×1012 fps is achieved. This implementation of CUSP is also highly flexible, allowing various combinations of imaging speeds and numbers of frames (several hundred up to 1000) to be readily deployed in diverse scientific studies, such as laser-induced transient birefringence, self-focusing, and filaments in dielectric media.

Ultralarge Pixel Array Photothermal Film based on 3D Self-Suspended Microbridge Structure for Infrared Scene Projection.

Infrared emitter is highly desirable for applications in infrared imaging and infrared stealth technology. It is also a core device in infrared scene generation. Light-driven photothermal film has attracted considerable interest due to its outstanding photothermal properties and easy fabrication. However, the existing photothermal films suffer from low photothermal conversion efficiency (PCE) as well as small sizes. The improvement of the PCE is usually achieved at the expense of dynamic frame rate. Here, this work designs and fabricates a photothermal film based on 3D self-suspended microbridge structure. Silicon (Si) microbridges are introduced into each microstructure to manipulate the thermal conductivity of the films. By optimizing the parameters of the Si microbridges, the high PCE and fast frame rate are both achieved. Moreover, the 3D structure microbridge film is 4-inch in diameter, forming an ultralarge array with over 2200 × 2200 pixels. Finally, a high PCE infrared scene projector is realized based on this photothermal film. A visible image is projected on the film, the 3D-microstructured photothermal film absorbs the visible light and emits an infrared image same as the visible one with high resolution and fast frame rate due to the excellent photothermal properties.