Target Scene Research Articles

Research on single-point fast three-dimensional concentration reconstruction of a smoke plume based on the static interference infrared imaging spectroscopy system

To meet the requirements of real-time and effectiveness of three-dimensional concentration telemetry of gas clouds and address the difficulty of fast three-dimensional concentration reconstruction of single instrument smoke plumes, the paper proposes a single-point fast three-dimensional concentration reconstruction of the smoke plume based on a static interference infrared Fourier transform imaging spectrometer (ISIFTS) by using the advantages of high throughput, large field of view, high stability, and rapid detection. Based on the scanning field switching characteristics of ISIFTS, a single instrument quickly scans the dual-field map data to obtain the three-dimensional point cloud of the target scene. Combined with the three-dimensional spatial parameters of the scene, a multi-dimensional re-projection algorithm is constructed based on the Gaussian plume continuous diffusion model to simulate the concentration-path-length product (CL) image, and the measured CL image is solved by analyzing the smoke plume image spectra data. The normalized cross-correlation fitting algorithm is used to perform the optimal matching of simulated-measured CL images, and then the three-dimensional concentration distribution of the plume scene is reconstructed based on the dimension mapping relationship. The field smoke plume telemetry experiment was designed and carried out, and the three-dimensional concentration distribution of plume gas at three emission points in the scene was obtained. The CO2 emission rates were calculated to be 13.572 kg / s, 12.225 kg / s, and 14.114 kg / s, respectively. The data are consistent with the emission rate of the coal-fired thermal power plant, which verifies the effectiveness of the single-point fast three-dimensional concentration reconstruction of the smoke plume method based on the ISIFTS system.

Automatic Method for Detecting Deformation Cracks in Landslides Based on Multidimensional Information Fusion

As cracks are a precursor landslide deformation feature, they can provide forecasting information that is useful for the early identification of landslides and determining motion instability characteristics. However, it is difficult to solve the size effect and noise-filtering problems associated with the currently available automatic crack detection methods under complex conditions using single remote sensing data sources. This article uses multidimensional target scene images obtained by UAV photogrammetry as the data source. Firstly, under the premise of fully considering the multidimensional image characteristics of different crack types, this article accomplishes the initial identification of landslide cracks by using six algorithm models with indicators including the roughness, slope, eigenvalue rate of the point cloud and pixel gradient, gray value, and RGB value of the images. Secondly, the initial extraction results are processed through a morphological repair task using three filtering algorithms (calculating the crack orientation, length, and frequency) to address background noise. Finally, this article proposes a multi-dimensional information fusion method, the Bayesian probability of minimum risk methods, to fuse the identification results derived from different models at the decision level. The results show that the six tested algorithm models can be used to effectively extract landslide cracks, providing Area Under the Curve (AUC) values between 0.6 and 0.85. After the repairing and filtering steps, the proposed method removes complex noise and minimizes the loss of real cracks, thus increasing the accuracy of each model by 7.5–55.3%. Multidimensional data fusion methods solve issues associated with the spatial scale effect during crack identification, and the F-score of the fusion model is 0.901.

Research on the Laser Scattering Characteristics of Three-Dimensional Imaging Based on Electro–Optical Crystal Modulation

In this paper, we construct a laser 3D imaging simulation model based on the 3D imaging principle of electro–optical crystal modulation. Unlike the traditional 3D imaging simulation method, this paper focuses on the laser scattering characteristics of the target scene. To accurately analyze and simulate the scattering characteristic model of the target under laser irradiation, we propose a BRDF (Bidirectional Reflectance Distribution Function) model fitting algorithm based on the hybrid BBO–Firefly model, which can accurately simulate the laser scattering distribution of the target at different angles. Finally, according to the fitted scattering characteristic model, we inverted the target imaging gray map. We used the laser 3D imaging restoration principle to reconstruct the 3D point cloud of the target to realize the laser 3D imaging of the target.

Improved Faster R-CNN Based on Multi-Scale Module for Detecting Weak Ground Targets in Remote Sensing Images

In order to optimize the detection capability of small and weak ground targets in remote sensing images, a remote sensing image ground weak target detection method based on multi-scale module improved Faster R-CNN is proposed. By adjusting the Region Proposal Network (RPN) and Region of Interest (RoI) pooling layers, the model is better suited for feature extraction and classification of small targets. Generate feature maps at multiple scales by introducing additional convolutional layers to process inputs at different resolutions, with each layer focusing on targets within a specific size range. These multi-scale features are integrated and input into RPN to provide richer target proposal characteristics. By using image transformation techniques such as scaling, cropping, and rotation, the training dataset is expanded to simulate different small target scenes, enhancing the model’s adaptability to changes in target size, direction, and environment. The experimental results show that the proposed method not only outperforms other models in accuracy, recall, and F1 score, but also maintains high consistency and stability across different datasets. Its significant advantage in mean accuracy (MAP) further proves its reliability and effectiveness in remote sensing applications.

Norm constraints pyramid for image dehazing

Dark channel prior-based methods have achieved remarkable performance for image dehazing. However, previous studies are mostly focused on the accuracy of the assumptions used in the target scenes, which incurs color distortion and brightness reduction when the models are used for real-world hazy images.We propose a norm constraints pyramid framework to improve the generalization performance of dehazing. First, a local color adaptive correction approach is devised to ascertain whether there is any color bias and thereafter rectify it automatically. Furthermore, multiple norm constraint methods are developed to improve the transmission and accomplish the first image removal. Finally, a non-linear enhancement method is created via this restriction that precisely modifies the brightness of an image.Through extensive experiments, we demonstrate that our framework establishes the new state-of- the-art performance for real-world dehazing, in terms of visual quality assessed by no-reference quality metrics as well as subjective evaluation and downstream task performance indicator.

Robust super-resolution classifier by nonlinear optics.

Spatial-mode projective measurements could achieve super-resolution in remote sensing and imaging, yet their performance is usually sensitive to the parameters of the target scenes. We propose and demonstrate a robust classifier of close-by light sources using optimized mode projection via nonlinear optics. Contrary to linear-optics based methods using the first few Hermite-Gaussian (HG) modes for the projection, here the projection modes are optimally tailored by shaping the pump wave to drive the nonlinear-optical process. This minimizes modulation losses and allows high flexibility in designing those modes for robust and efficient measurements. We test this classifier by discriminating one light source and two sources separated well within the Rayleigh limit without prior knowledge of the exact centroid or brightness. Our results show a classification fidelity of over 80% even when the centroid is misaligned by half the source separation, or when one source is four times stronger than the other.

Lightweight Substation Equipment Defect Detection Algorithm for Small Targets.

Substation equipment defect detection has always played an important role in equipment operation and maintenance. However, the task scenarios of substation equipment defect detection are complex and different. Recent studies have revealed issues such as a significant missed detection rate for small-sized targets and diminished detection precision. At the same time, the current mainstream detection algorithms are highly complex, which is not conducive to deployment on resource-constrained devices. In view of the above problems, a small target and lightweight substation main scene equipment defect detection algorithm is proposed: Efficient Attentional Lightweight-YOLO (EAL-YOLO), which detection accuracy exceeds the current mainstream model, and the number of parameters and floating point operations (FLOPs) are also advantageous. Firstly, the EfficientFormerV2 is used to optimize the model backbone, and the Large Separable Kernel Attention (LSKA) mechanism has been incorporated into the Spatial Pyramid Pooling Fast (SPPF) to enhance the model's feature extraction capabilities; secondly, a small target neck network Attentional scale Sequence Fusion P2-Neck (ASF2-Neck) is proposed to enhance the model's ability to detect small target defects; finally, in order to facilitate deployment on resource-constrained devices, a lightweight shared convolution detection head module Lightweight Shared Convolutional Head (LSCHead) is proposed. Experiments show that compared with YOLOv8n, EAL-YOLO has improved its accuracy by 2.93 percentage points, and the mAP50 of 12 types of typical equipment defects has reached 92.26%. Concurrently, the quantity of FLOPs and parameters has diminished by 46.5% and 61.17% respectively, in comparison with YOLOv8s, meeting the needs of substation defect detection.

Sequential Two-Mode Fusion Underwater Single-Photon Lidar Imaging Algorithm

Aiming at the demand for long-range and high-resolution imaging detection of small targets such as submerged submarine markers in shallow coastal waters, research on single-photon lidar imaging technology is carried out. This paper reports the sequential two-mode fusion imaging algorithm, which has a strong information extraction capability and can reconstruct scene target depth and reflection intensity images from complex signal photon counts. The algorithm consists of four steps: data preprocessing, extremely large group value estimation, noise sieving, and total variation smoothing constraints to image the target with high quality. Simulation and test results show that the imaging performance and imaging characteristics of the method are better than the current high-performance first-photon group imaging algorithm, indicating that the method has a great advantage in sparse photon counting imaging, and the method proposed in this paper constructs a clear depth and reflectance intensity image of the target scene, even in the 50,828 Lux ambient strong light and strong interference, the 0.1 Lux low-light environment, or the underwater high-attenuation environment.

Fast phase matching based on order partition with sub-pixel accuracy

Structured light is a method of 3D imaging that involves projecting a sequence of striped or structured patterns onto a target scene and then, based on the warping of stripes across a target surface, a camera can reconstruct the target. For practical reasons relating to changing lens parameters for the projector, binocular structured light improves over monocular structured by performing triangulation between two cameras, using the striped patterns to solve the correspondence matching problem; however, the speed at which these correspondences are found is still an active area of research. And in this paper, we propose a fast-matching algorithm that enhances phase labeling and introduces the concept of triangle plane interpolation based on linear interpolation. The experimental results demonstrate that our method achieves a noticeable speed improvement in computational efficiency and a stable improvement in accuracy as compared to the phase matching along epipolar line pointwisely based on look-up tables.

Rethinking of Underwater Image Restoration Based on Circularly Polarized Light

Polarimetric imaging technology plays a crucial role in de-scattering, particularly in the field of underwater image restoration. Circularly polarized light (or the underlying circular polarization memory effect) has been proven to better preserve the polarization characteristics of detected light. Utilizing circularly polarized light as illumination can further enhance the effectiveness of polarization de-scattering techniques. After rethinking the advantages of circularly polarized light, this paper proposes a new method for underwater polarimetric imaging restoration that leverages the pre-processing of polarized sub-images and the correlation of polarization characteristics (i.e., the angle of polarization and degree of polarization). Additionally, to address the challenge of selecting scattering light parameters due to uneven light fields in target scenes, an intensity adjustment factor search algorithm is designed. This algorithm eliminates the need for the manual selection of scattering light parameters, effectively solving the problem of uneven illumination in restoration results. A series of experiments demonstrate that, compared to traditional algorithms, the proposed method offers superior detail restoration and higher robustness.

Enhancing three-source cross-modality image fusion with improved DenseNet for infrared polarization and visible light images

The fusion of multi-modal images to create an image that preserves the unique features of each modality as well as the features shared across modalities is a challenging task, particularly in the context of infrared (IR)-visible image fusion. In addition, the presence of polarization and IR radiation information in images obtained from IR polarization sensors further complicates the multi-modal image-fusion process. This study proposes a fusion network designed to overcome the challenges associated with the integration of low-resolution IR, IR polarization, and high-resolution visible (VIS) images. By introducing cross attention modules and a multi-stage fusion approach, the network can effectively extract and fuse features from different modalities, fully expressing the diversity of the images. This network learns end-to-end mapping from sourced to fused images using a loss function, eliminating the need for ground-truth images for fusion. Experimental results using public datasets and remote-sensing field-test data demonstrate that the proposed methodology achieves commendable results in qualitative and quantitative evaluations, with gradient based fusion performance QAB/F, mutual information (MI), and QCB values higher than the second-best values by 0.20, 0.94, and 0.04, respectively. This study provides a comprehensive representation of target scene information that results in enhanced image quality and improved object identification capabilities. In addition, outdoor and VIS image datasets are produced, providing a data foundation and reference for future research in related fields.

Audio-guided implicit neural representation for local image stylization

We present a novel framework for audio-guided localized image stylization. Sound often provides information about the specific context of a scene and is closely related to a certain part of the scene or object. However, existing image stylization works have focused on stylizing the entire image using an image or text input. Stylizing a particular part of the image based on audio input is natural but challenging. This work proposes a framework in which a user provides an audio input to localize the target in the input image and another to locally stylize the target object or scene. We first produce a fine localization map using an audio-visual localization network leveraging CLIP embedding space. We then utilize an implicit neural representation (INR) along with the predicted localization map to stylize the target based on sound information. The INR manipulates local pixel values to be semantically consistent with the provided audio input. Our experiments show that the proposed framework outperforms other audio-guided stylization methods. Moreover, we observe that our method constructs concise localization maps and naturally manipulates the target object or scene in accordance with the given audio input.

UDANet: An unsupervised domain adaptive vehicle density estimation network based on joint adversarial learning

Unsupervised domain adaptive vehicle density estimation aims to transfer the knowledge learned from the labeled source domain to the unlabeled target domain, which has received extensive attention due to its practicality and effectiveness. However, the existing methods often rely on a large amount of expensive labeled data, and ignore the large differences between different data domains, making the network trained in one scene difficult to apply to the target scene. To solve these problems, this paper proposes an unsupervised domain adaptive vehicle density estimation network based on joint adversarial learning (UDANet), which effectively extends the model to unlabeled target domain data by adding two domain adaptive components to the pre-trained backbone. Specifically, a multi-level feature discriminator is designed to align vehicle features from the image level and refine features from the pixel level by joint adversarial learning, thus achieving cross-domain transfer of features from the source domain to the target domain. In addition, to address the shared convolutional structure that incorrectly highlights irrelevant information, a feature correction module is designed to improve the adaptability of the network to target domain data by re-modeling the source and target data according to channel independence. Several well-known backbones are used to prove the superiority of the backbone and the mobility of the two domain adaptive components. The domain adaptation experimental results on four vehicle datasets show that UDANet achieves high estimation accuracy compared to current state-of-the-art unsupervised methods, and is suitable for the task of vehicle density estimation in complex scenes.

Freeform-surface-based optical design of a broadband compressive spectral imager with co-aperture coding

Compressive spectral imaging (CSI) is an advanced computational imaging approach to reconstruct the three-dimensional (3D) spatio-spectral data cube of a target scene through a single or a few snapshots. However, limited by the response range of the image detector, the existing CSI systems mostly work within narrow spectral bands, such as the visible or shortwave-infrared (SWIR) spectral band. The work band of the CSI system constrains the detection capacity for the targets under complex environments (such as rain, snow, haze, etc.). In addition, most of the current CSI prototypes lack engineering design for practical applications. This paper develops a novel, to the best of our knowledge, optical design scheme of a broadband CSI system with co-aperture coding to simultaneously realize visible multi-spectral imaging (10 channels) and SWIR super-resolution imaging (16×). The freeform surfaces are used to design the front-end reflective objective lens, thus significantly improving the image quality and spatial modulation precision of the system. By means of performance evaluation and tolerance analysis, excellent image quality and manufacturability of the proposed system are demonstrated.

Spatial-spectral encoding and dictionary optimization in compressive single-pixel hyperspectral imaging based on mutual coherence minimization

A single-pixel detector based hyperspectral system provides an effective way to obtain the spatial-spectral information of target scenes. However, complex spectral dispersion and the substantial number of measurements not only increase the complexity of the system but also decrease the sampling efficiency and the reconstruction accuracy. In this paper, we propose a compressive sensing (CS) theory based single-pixel hyperspectral imaging system. Based on structured illumination, the spatial information is modulated by binary spatial patterns displayed on a liquid crystal on silicon (LCoS), while polarizing elements at specific angles, acting as a serious of filters, modulate the spectral dimension, effectively avoiding spectral dispersion. In terms of sampling efficiency, the application of CS significantly decreases the number of measurements required compared to the Nyquist-Shannon sampling theorem. Besides, to improve the reconstruction accuracy, mutual coherence minimization is employed to optimize the pre-trained dictionary, spatial patterns and filters. Furthermore, a two-step encoding method based on macro-pixel segmentation is proposed to address the issue of low resolution constrained by the size of the dictionary. Compared to the unoptimized system and dictionary, the proposed method achieves more accurate reconstruction results in both spectral and spatial dimensions. This work may provide opportunities for high-resolution single-pixel hyperspectral imaging systems based on CS.

Application of Gaussian Model and Deep Learning Encoder-Decoder Algorithm for Single-Image Reflection Removal

Images of target scenes shot through clear, reflective materials like glass are frequently interfered by unwanted reflection scenes which often overlaid on top of the targeted scenes. This, has constantly degrades the quality of the captured images and affects their subsequent analyses. While cognitively, distinguishing a recognizable object from its reflection in a picture is not difficult for humans, it is highly difficult and more complex in computer vision due to the ill-posed nature of the problem. In this research an enhanced single-image reflection removal model was developed by combining Gaussian filter and deep learning encoder-decoder for effective performance. While the Gaussian filter denoises the reflection-contaminated image, the encoder-decoder network learns the features of the image to produce reflection-free image. The proposed network is an end-to-end trained network with three losses. The experimental findings showed that the proposed model out-performed several state-of-the-art methods both qualitatively and quantitatively on five different datasets.

Palm vein imaging using a polarization-selective metalens with wide field-of-view and extended depth-of-field

Metalens is a flat, lightweight optical element that may replace traditional bulky refractive lenses and other components in imaging systems. However, a number of critical challenges still remain for most metalenses demonstrated to date, including limited field-of-view (FOV), depth-of-field (DOF), and working bandwidth, which restrict their use in a real-world application. Here, we propose and experimentally demonstrate a compact imaging system equipped with a metalens that simultaneously features a wide FOV of 140° and an extended DOF ranging from 33 to 150 mm. The metalens further allows polarization selectivity, which can be used to remove unwanted surface reflection of the target scene. Using a narrowband vertical-cavity surface-emitting laser for illumination, we show that the system is well-suited for near-infrared palm vein imaging, an emerging modality for biometric identification. The metalens-integrated imaging system provides uncompromised performance with a greatly simplified form factor compared to a traditional system, which may also be adopted for other applications such as depth sensing and endoscopy.

Super-Resolution Image Reconstruction of Wavefront Coding Imaging System Based on Deep Learning Network

Wavefront Coding (WFC) is an innovative technique aimed at extending the depth of focus (DOF) of optics imaging systems. In digital imaging systems, super-resolution digital reconstruction close to the diffraction limit of optical systems has always been a hot research topic. With the design of a point spread function (PSF) generated by a suitably phase mask, WFC could also be used in super-resolution image reconstruction. In this paper, we use a deep learning network combined with WFC as a general framework for images reconstruction, and verify its possibility and effectiveness. Considering the blur and additive noise simultaneously, we proposed three super-resolution image reconstruction procedures utilizing convolutional neural networks (CNN) based on mean square error (MSE) loss, conditional Generative Adversarial Networks (CGAN), and Swin Transformer Networks (SwinIR) based on mean absolute error (MAE) loss. We verified their effectiveness by simulation experiments. A comparison of experimental results shows that the SwinIR deep residual network structure based on MAE loss optimization criteria can generate more realistic super-resolution images with more details. In addition, we used a WFC camera to obtain a resolution test target and real scene images for experiments. Using the resolution test target, we demonstrated that the spatial resolution could be improved from 55.6 lp/mm to 124 lp/mm by the proposed super-resolution reconstruction procedure. The reconstruction results show that the proposed deep learning network model is superior to the traditional method in reconstructing high-frequency details and effectively suppressing noise, with the resolution approaching the diffraction limit.

Single-pixel compressive imaging via single photon counting

Single-pixel compressive imaging reconstructs a target scene with many pixels by using a single-pixel detector to measure the power variations as small sequences of sampling patterns are applied. While it boasts remarkable capabilities, its practical applications remain a challenge in the photon-starved regime where signal-to-noise is low. To address this challenge, we propose to combine quantum parametric mode sorting (QPMS) and deep neural networks (DNN) to overcome low signal-to-noise for faithful image construction. We benchmark our approach in a telecom-LiDAR system against that using direct photon counting detection. Our results show that with only 25 sampling patterns (corresponding compression ratio ∼0.043%), QPMS plus DNN give structural similarity index measure and peak signal-to-noise ratio on average above 22 dB and 0.9, respectively, much higher than those with direct detection (DD). The details of our targets from QPMS are more clearly compared with from DD. Notably, such high performance is sustained even in the presence of 500 times stronger in-band background noise, while DD fails. The high efficiency and robust noise rejection promise potential applications in various fields, especially in photon-starving scenarios.

Encoding images to 3D by sequential model with single target feature extractor

Assisted driving is a necessary way to realize autonomous driving, in which bird's-eye-view (BEV) is an ideal solution to perceive the targets around the body, i.e., the information acquired by the sensors of the body is extracted and semantic features are integrated into the BEV plane for downstream tasks such as target detection, scene segmentation, and path planning, etc. BEV-based pure vision target detection refers to the use of ordinary cameras without relying on other sensors to perceive the targets around the body. BEV-based pure visual target detection, on the other hand, refers to the use of ordinary cameras to perceive targets around the body without relying on other sensors, and Lift Splat Shoot (LSS) is a more typical solution among the existing schemes. Since the information obtained by each camera in assisted driving is always continuous, incorporating the temporal information into the model can achieve better detection results. We design the model (Sequential and Single Target based LSS) SSTL, and the experiment proves that our model has a certain performance improvement based on the original model.