High Resolution Image Processing Research Articles

Advanced Method for Improving Marine Target Tracking Based on Multiple-Plot Processing of Radar Images

Advancements in technology have led to the development of high-resolution radars that provide highly detailed images of targets over a wide field of view. These radar images can significantly improve filtering and tracking accuracy, especially in marine environments. However, traditional methods like the binary and barycentric methods are inadequate, as they do not capture critical information for tracking targets, such as direction. Therefore, in this article, a new method for improving the estimation of target characteristics to improve tracking accuracy based on the processing of high-resolution radar images is proposed. The proposed method consists of three modules. Firstly, the radar images of the target are decomposed into layers to determine local maximum regions and to estimate target characteristics such as reflected energy and area and the centroids of plots. In the second module, the plots are grouped using a fuzzy logic system. The inputs of the fuzzy logic system include the above-estimated parameters of the targets. The output is the chance that the plot is at the center of the target. The plots with the highest chances are considered target centers, and the other plots are grouped into their respective target. At the end, the true target center is recalculated. This process is called modified fuzzy C-means (FCM-M). In the last stage, the estimated target center coordinates are fed into a Kalman filter (KF) to solve filtering and tracking problems. The performance is evaluated using a measured radar dataset. The experimental results show that the proposed method performs better than traditional methods based on binary image processing. Additionally, the proposed method offers extra information about the targets, such as their direction, the energy of each reflected part, and the area, which traditional methods does not provide.

Extraction of offshore aquaculture areas from Gaofen-2 remote sensing imagery based on deep learning

ABSTRACT Extracting offshore aquaculture areas from high-resolution remote sensing images is significant for rational planning and dynamic management of aquaculture. However, the complex and changeable characteristics of aquaculture areas will lead to difficulty during high-resolution image processing. In this paper, an improved U-Net deep learning model is proposed to extract offshore aquaculture areas from high-resolution images. This model is improved by the pyramid pooling module (PPM) and up-sampling structure based on the original U-Net model. PPM and the up-sampling structure can better mine semantic and position information in high-resolution images, reduce the interference and adhesion of information in images, and improve the extraction accuracy of unclear aquaculture areas. Based on Gaofen-2 remote sensing image data of Rongcheng Bay in the eastern waters of Weihai City, Shandong Province, the extraction of offshore aquaculture areas is studied. The results show that the F1 score and MIoU of the proposed network model are 93.63% and 87.93%, respectively. Compared with several commonly used deep learning models, the improved U-Net model has a better extraction effect.

New geographic distribution record of Dactylogyrus labei Musselius and Gusev (1976) (Platyhelminthes, Monogenea) of Labeo rohita (Teleostei: Cyprinidae) from Mizoram, northeast India

Dactylogyrus labei is a dactylogyrid monogenean parasite originally described by Musselius and Gusev (1976) from Labeo rohita (Hamilton, 1822) (Teleostei: Cyprinidae) in two alternative forms: a typical from Lucknow, north India and an atypical from Kalyani, east India. Fresh specimens of D. labei were collected from Mizoram in northeast India, and examined morphologically using high-resolution digital image processing and analysis. The recovered monogeneans most closely resembled the atypical form of D. labei, with only minor differences, indicating conspecificity. The atypical form of D. labei is distinguished from other species of the genus by the following combination of characters: the dorsal bar wing-shaped, with a fine membrane on the anterior margin and a fenestration in the middle; the ventral bar casing nail-shaped with two small antero-lateral processes (3-rayed); the male copulatory tube loosely S-shaped, slender, with an inflated initial part; the accessory piece proximally tubular, distally a complex of multi-layered sheath guiding the copulatory tube, and the vagina tubular, with one small loop and vaginal pore surrounded with a flap-like structure. This study expands the geographical distribution of D. labei while improving its morphological descriptions.

Clinical and Educational Aspects of Neuroimaging in Microsurgery of the Posterior Fossa: A Comprehensive Review.

The clinical and educational value of modern high-resolution magnetic resonance imaging (MRI) and image processing in neurovascular diseases of the posterior fossa with regard to preoperative planning and intraoperative comparison with the actual anatomical situation was consecutively evaluated. Patients with trigeminal neuralgia (TN), hemifacial spasm (HFS), glossopharyngeal neuralgia (GN) and arterial hypertension (HTN) were analyzed. The high-resolution MRI data was segmented and visualized three-dimensionally using computer graphics methods. New anatomical insights were gained, such as the classification of neurovascular compression (NVC) in HFS and GN. It was also possible to visualize the pathognomonic cerebrospinal fluid signal in patients with TN for the first time. Using the new imaging methods, pregnant hypertensive patients were examined and the existence of NVC was confirmed for the first time, and the findings were compared to other studies dealing with NVC syndromes. This review gives an overview on the established methods of neuroimaging and image processing of neurovascular structures in the posterior fossa with the focus on clinical and educational aspects.

Image Deblurring Based on Efficient Transformer and Multi-scale CNN

The study of image deblurring techniques in dynamic scenes is a high-profile research direction. Recently, given the excellence of Convolutional Neural Networks (CNNs) in extracting feature information, it has become a common and effective practice to utilize them for blurred image restoration work. However, CNN can only model local information and has a limited receptive field, which inhibits the deblurring effect. Transformer can model global information, so it can be combined with CNN to expand the receptive field and enhance the deblurring effect. Unfortunately, as the spatial resolution of the input image increases, the computational complexity of the Transformer increases dramatically, showing a trend of square-level growth, which makes it difficult to cope with the task of processing high-resolution images. To address the above problems, this paper proposes an image deblurring network based on efficient Transformer and multi-scale CNN called ET-MIMO-UNet. The local spatial features are extracted using multi-scale CNN and embedded into the global characteristics of the Transformer, modelling both local and global information. To solve the problem of difficult training due to large image size and to improve the computational efficiency, an efficient Transformer layer (ETL) is designed, which contains a multi-dconv head transposed attention (MHTA) and a gated-dconv feed-forward network(GFFN). In addition, a multi-layer feature fusion block (MFFB) is introduced to fuse full-scale features and reduce feature loss. On the GoPro test dataset, compared with the MIMO-UNet base network, the PSNR of the three models of ET-MIMO-UNet is improved by 0.39dB, 0.54dB, and 0.66dB, respectively; ET-MIMO-UNet reduces the number of parameters by half compared with MPRNet. The experimental data fully proved that the method demonstrated a significant processing effect in coping with the image-blurring problem in dynamic scenes.

Evaluation of multi-camera images in different SfM-MVS based photogrammetric software and comparison of digital products in generating 3D city models

High-resolution 3D city modeling is generated using the oblique and vertical (nadir) imaging features of multi-camera systems. There are several SfM-MVS based photogrammetric software available. They enable high-resolution image processing and matching and acquire successful outcomes. The orientation of large photogrammetric blocks consisting of multiple sets of images and the accuracy of the 3D city models to be obtained vary according to the ability of the software using these approaches to solve large image matrix problems. In this study, a 3D city model was produced with vertical and oblique images obtained with UltraCam Osprey Mark 3 Premium camera in a selected application area in Kütahya city center. The main purpose of the study is to evaluate the point cloud, textured mesh model, digital elevation model, true orthophoto products and vectorial roof detail drawings via utilizing the vertical + oblique images with different SfM-MVS based photogrammetric software and to compare the results.

High-Resolution Image Processing and Spatiotemporal Data Transmission System Based on GPU Acceleration

With the development of information technology and the increasing demand for data processing, the serial mode of the central processing unit (CPU) is difficult to efficiently transmit large-scale spatiotemporal data, and the processing effect for high-resolution images is not good. This paper designed a high-resolution image processing and spatiotemporal data transmission system based on graphics processing unit (GPU) acceleration to improve the processing efficiency of large-scale spatiotemporal data. In this paper, traffic spatiotemporal data was taken as an example for analysis. Large-scale traffic image data was collected by road monitoring equipment, and image compression was performed on the collected image. Fourier transform was used to eliminate image data redundancy, and GPU-accelerated parallel processing was used to achieve fast image defogging and data transmission. This paper selected 2TB of traffic spatiotemporal data with image resolutions of 540P, 720P, 1080P, 1440P, and 2160P. GPU acceleration was performed using the Compute Unified Device Architecture (CUDA). In images with a resolution of 2160P, the processing time for CPU and GPU acceleration was 2900ms and 28ms, respectively, with an acceleration ratio of 103.6. A high-resolution image processing and spatiotemporal data transmission system based on GPU acceleration can improve the efficiency of traffic spatiotemporal data processing and have excellent concurrent processing capabilities.

Application of infrared image detection based on high-resolution image processing in motion recognition

Application of infrared image detection based on high-resolution image processing in motion recognition

Ensemble Deep Learning for Automated Damage Detection of Trailers at Intermodal Terminals

Efficient damage detection of trailers is essential for improving processes at inland intermodal terminals. This paper presents an automated damage detection (ADD) algorithm for trailers utilizing ensemble learning based on YOLOv8 and RetinaNet networks. The algorithm achieves 88.33% accuracy and an 81.08% F1-score on the real-life trailer damage dataset by leveraging the strengths of each object detection model. YOLOv8 is trained explicitly for detecting belt damage, while RetinaNet handles detecting other damage types and is used for cropping trailers from images. These one-stage detectors outperformed the two-stage Faster R-CNN in all tested tasks within this research. Furthermore, the algorithm incorporates slice-aided hyper inference, which significantly contributes to the efficient processing of high-resolution trailer images. Integrating the proposed ADD solution into terminal operating systems allows a substantial workload reduction at the ingate of intermodal terminals and supports, therefore, more sustainable transportation solutions.

Rendering lineament induced stream alignment in Upper Krishna Basin, India: a geospatial approach

Lineaments play an important role in drainage development, stream alignment and groundwater recharge. These are traced as surface or subsurface feature. A linear feature that is associated with dislocation and deformation is known as lineament. The present study was aimed to identify the influence of lineaments extracted from satellite images in the Upper Krishna River basin in India. Objective set for the present study is to characterize and analyse the spatial organization of lineaments from ASF-DEM and sentinel-2 satellite data which may help to determine the role of lineaments in groundwater flow and to identify groundwater potential zones. High resolution spatial data allows determining the linear features for considerable distance and play an important role to identify mineral potential areas (Mohammadpour et al. in Geotectonics 54:366–382, 2020). Comprehensive understanding of regional lineament maps is important in terms of their prospectivity for mineral exploration (Richards in SEG Newsl 42:1–20, 2000). Remote sensing datasets are considered the best option when using image enhancement techniques for extracting lineaments. Lineament studies are useful for groundwater and mineral exploration and also in the field of engineering geology (Anbazhagan in Bhu-Jal News 8:8–12, 1994; Anbazhagan et al. in New approaches to characterizing groundwater flow, 2001. http://dspace.library.iitb.ac.in/xmlui/handle/10054/16176, http://hdl.handle.net/100/2781) more generally in the geomorphological studies like forms of surface features and structural geological fields also in physical setting and control over the rocky terrain. The availability of high-resolution satellite data and image processing techniques have rendered it further convenient to map lineaments. In the present study, lineaments were extracted from Sentinel-2 images with a resolution of 10 m, using various image processing techniques. A total of 1314 lineaments were extracted from the study area with a total length of 3983.44 km. The analysis of the extracted lineaments revealed that the lineament density was higher in the upper reaches of the basin, where the undulating hilly region which is located in Western Ghats. This finding implied that these regions have a high structural deformation and a higher groundwater infiltration potential. Moreover, 15% of the total stream length was observed to be influenced by the lineaments. The maximum influence of lineaments was observed in the source region. The lineament extraction results of the present study would assist in understanding the geomorphology of this region and the structural control on the streams and groundwater potential zones, particularly as a contribution to water resource management in this region.

High-Resolution Image Processing of Probe-Based Confocal Laser Endomicroscopy Based on Multistage Neural Networks and Cross-Channel Attention Module

Probe-based confocal laser endomicroscopy (pCLE) is a subcellular in vivo imaging technique that generates diagnostic images revealing malignant structural modifications in epithelial tissues. In the clinical diagnosis of probe confocal laser endomicroscopy (pCLE), the image background generally has the problems of dynamic blur or information loss, which is not conducive to achieving high-resolution and clear pCLE imaging. In recent years, deep learning technology has achieved remarkable results in image deblurring. For the task of recovering high-resolution pCLE images, the current methods still suffer from the following drawbacks: it is difficult to choose a strategy to make CNN converge at a deeper level and mainstream methods cannot handle the complex balance between spatial details and high-level feature information well when reconstructing clear images. In order to solve the problem, we propose a new cross-channel attention, multistage, high-resolution pCLE image deblurring structure. This methodology improves the supervised attention mechanism, enhances the ability of feature extraction and fusion capabilities, and improves the quality of image deblurring by adding cross-channel attention module (CAM) into the multistage neural networks’ architecture. The experimental results show that the average peak signal-to-noise ratio (PSNR) of the proposed model on the dataset is as high as 29.643 dB, and the structural similarity (SSIM) reaches 0.855. This method is superior to the prior algorithms in the visualization of recovered images, and the edge and texture details of the restored pCLE images are clearer.

High-Fidelity and Efficient Pluralistic Image Completion with Transformers.

Image completion has made tremendous progress with convolutional neural networks (CNNs), because of their powerful texture modeling capacity. However, due to some inherent properties (e.g., local inductive prior, spatial-invariant kernels), CNNs do not perform well in understanding global structures or naturally support pluralistic completion. Recently, transformers demonstrate their power in modeling the long-term relationship and generating diverse results, but their computation complexity is quadratic to input length, thus hampering the application in processing high-resolution images. This paper brings the best of both worlds to pluralistic image completion: appearance prior reconstruction with transformer and texture replenishment with CNN. The former transformer recovers pluralistic coherent structures together with some coarse textures, while the latter CNN enhances the local texture details of coarse priors guided by the high-resolution masked images. To decode diversified outputs from transformers, auto-regressive sampling is the most common method, but with extremely low efficiency. We further overcome this issue by proposing a new decoding strategy, temperature annealing probabilistic sampling (TAPS), which firstly achieves more than 70× speedup of inference at most, meanwhile maintaining the high quality and diversity of the sampled global structures. Moreover, we find the full CNN architecture will lead to suboptimal solutions for guided upsampling. To render more realistic and coherent contents, we design a novel module, named texture-aware guided attention, to concurrently consider the procedures of texture copy and generation, meanwhile raising several important modifications to solve the boundary artifacts. Through dense experiments, we found the proposed method vastly outperforms state-of-the-art methods in terms of four aspects: 1) large performance boost on image fidelity even compared to deterministic completion methods; 2) better diversity and higher fidelity for pluralistic completion; 3) exceptional generalization ability on large masks and generic dataset, like ImageNet. 4) Much higher decoding efficiency over previous auto-regressive based methods.

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition.

Vision Transformers have been the most popular network architecture in visual recognition recently due to the strong ability of encode global information. However, its high computational cost when processing high-resolution images limits the applications in downstream tasks. In this paper, we take a deep look at the internal structure of self-attention and present a simple Transformer style convolutional neural network (ConvNet) for visual recognition. By comparing the design principles of the recent ConvNets and Vision Transformers, we propose to simplify the self-attention by leveraging a convolutional modulation operation. We show that such a simple approach can better take advantage of the large kernels ( ≥ 7×7) nested in convolutional layers and we observe a consistent performance improvement when gradually increasing the kernel size from 5×5 to 21×21. We build a family of hierarchical ConvNets using the proposed convolutional modulation, termed Conv2Former. Our network is simple and easy to follow. Experiments show that our Conv2Formeroutperforms existent popular ConvNets and vision Transformers, like Swin Transformer and ConvNeXt in all ImageNet classification, COCO object detection and ADE20k semantic segmentation. Our code is available at https://github.com/HVision-NKU/Conv2Former.

An Efficient Multiscale Spatial Rearrangement MLP Architecture for Image Restoration.

The effective use of long-range information can yield improved network performance, which is very important for image restoration. Although local window-based models have linear complexity and can be feasibly applied to process high-resolution images, a single-scale window has a limited receptive field and is less efficient for encoding long-range context information. To address this issue, this paper presents a single-stage multiscale spatial rearrangement multilayer perceptron (MSSR-MLP) architecture that can obtain information at different scales within a local window. Specifically, we propose a simple and efficient spatial rearrangement module (SRM) that moves information outside the local window to the inside of the local window so that long-range dependencies can be modeled using only a window-based fully connected (FC) layer. The SRM can extend the local receptive field of a window-based FC layer without introducing additional parameters and FLOPs. Utilizing several spatial rearrangement modules with different step sizes, we design an efficient multiscale spatial rearrangement MLP architecture for image restoration. This design aggregates multiscale information to achieve improved restoration quality while maintaining a low computational cost. Extensive experiments conducted on several image restoration tasks demonstrate the efficiency and effectiveness of our method. For example, it requires only ~4.3% of the FLOPs needed by SwinIR for Gaussian gray image denoising, ~13.9% of the FLOPs needed by C2 PNet for single-image dehazing and ~18.9% of the FLOPs needed by MAXIM for single-image motion deblurring but achieves better performance on each of these restoration tasks.

Sediment imaging and factorized Haar wavelet transformation techniques for characterizing particle size distributions of granular soils

The particle size distribution (PSD) significantly influences various mechanical behaviors of granular soils. Advances in automation and camera technology allow for a rapid, green, and accurate PSD determination. This study develops a new sediment imaging system, mainly consisting of a sediment column filled with water and a line-scan camera driven by programmable logic controller, which can automatically capture high-resolution soil segregation images. The soil particles settle through the sediment column and segregate by size, which is scanned to generate sediment images. Naturally, the coarsest particles settle at the bottom of the image, while the finest particles are located at the top. Therefore, the analysis of the sediment image enables the determination of PSD. The higher sediment column, despite of poor operationality, yields better particle segregation quality and thus the more accurate PSD result. This study proposed a factorized Haar wavelet transformation (FHWT) algorithm that can account for less perfect segregation of particles by size, enabling to shorten the height of sediment column. The proposed hardware and FHWT are evaluated by six natural sands with different colors, internal textures, and geological origins. An excellent match between image-based PSDs and sieving-based PSDs are observed. Based on FHWT, four sediment columns with different heights of 2.1 m, 1.5 m, 1.0 m, 0.5 m are tested. The results indicate that columns higher than 1.0 m yield acceptable particle segregation results. Furthermore, the proposed FHWT is also adaptable to process high-resolution images captured by advanced cameras.

Booster: A Benchmark for Depth From Images of Specular and Transparent Surfaces.

Estimating depth from images nowadays yields outstanding results, both in terms of in-domain accuracy and generalization. However, we identify two main challenges that remain open in this field: dealing with non-Lambertian materials and effectively processing high-resolution images. Purposely, we propose a novel dataset that includes accurate and dense ground-truth labels at high resolution, featuring scenes containing several specular and transparent surfaces. Our acquisition pipeline leverages a novel deep space-time stereo framework, enabling easy and accurate labeling with sub-pixel precision. The dataset is composed of 606 samples collected in 85 different scenes, each sample includes both a high-resolution pair (12 Mpx) as well as an unbalanced stereo pair (Left: 12 Mpx, Right: 1.1 Mpx), typical of modern mobile devices that mount sensors with different resolutions. Additionally, we provide manually annotated material segmentation masks and 15 K unlabeled samples. The dataset is composed of a train set and two test sets, the latter devoted to the evaluation of stereo and monocular depth estimation networks. Our experiments highlight the open challenges and future research directions in this field.

Estimation of Left Ventricular Ejection Fraction Based on Swin_Uniform

A new left ventricular ejection index prediction method was proposed by introducing a block attention mechanism, which helps to reduce overfitting problems and improve the power of the model to process new different samples. Swin Transformer is a Transformer architecture that retains the modeling advantages of self attention mechanism. Its hierarchical design has both global and local modeling capabilities, and can output multi resolution feature maps, which is suitable for EchoCoTr's need to extract multi-scale information from time series medical images. Meanwhile, the resources required of Swin Transformer at runtime are linearly related to images to be processed, which reduces computational complexity and is suitable for processing high-resolution sequence images in EchoCoTr[1]. Swin Transformer has better results than Transformer in both classification and detection tasks, which just goes to show that it is somewhat of a great application in the field of vision. This is beneficial for EchoCoTr's understanding of cardiac ultrasound sequences. Therefore, in the original UniFormer model structure of the EchoCoTr[1] model, a new Swin Transformer Block is added. The new model has stronger modeling capabilities. This means that better results can be obtained under the same computing resources.

CTFCD: Channel transformer based on full convolutional decoder for single image deraining

Although convolutional neural network and visual transformer have been successfully applied in various field of computer vision, there is little work combining them to construct an efficient network model to solve image deraining tasks. Convolutional neural network is utilized to extract the features from each region, while visual transformer is utilized to extract the context information between local features. Due to the limitations in computational resources and processing time, it is difficult for visual transformer to process high-resolution images, which hinders the application of visual transformer in devices with limited hardware resources. The purpose of this article is to utilize the advantages of to design a lightweight encoder-decoder network for real-time image deraining. Firstly, a novel channel Transformer module is designed to obtain global contextual information, where deep separable convolution is utilized to extract multi-scale local features and a Transformer encoder is constructed by stacking Transformer modules. Secondly, a decoder based on a fully convolution is designed to adopt mask attention and inverted bottleneck convolution to achieve progressive feature fusion and feature reconstruction, which significantly reduces computational complexity and memory requirement. A large number of experimental results have verified that the proposed method has superior performance compared with other state-of-the art methods, while the computational cost and parameter quantity are much smaller than those of similar methods.

RCKD: Response-Based Cross-Task Knowledge Distillation for Pathological Image Analysis.

We propose a novel transfer learning framework for pathological image analysis, the Response-based Cross-task Knowledge Distillation (RCKD), which improves the performance of the model by pretraining it on a large unlabeled dataset guided by a high-performance teacher model. RCKD first pretrains a student model to predict the nuclei segmentation results of the teacher model for unlabeled pathological images, and then fine-tunes the pretrained model for the downstream tasks, such as organ cancer sub-type classification and cancer region segmentation, using relatively small target datasets. Unlike conventional knowledge distillation, RCKD does not require that the target tasks of the teacher and student models be the same. Moreover, unlike conventional transfer learning, RCKD can transfer knowledge between models with different architectures. In addition, we propose a lightweight architecture, the Convolutional neural network with Spatial Attention by Transformers (CSAT), for processing high-resolution pathological images with limited memory and computation. CSAT exhibited a top-1 accuracy of 78.6% on ImageNet with only 3M parameters and 1.08 G multiply-accumulate (MAC) operations. When pretrained by RCKD, CSAT exhibited average classification and segmentation accuracies of 94.2% and 0.673 mIoU on six pathological image datasets, which is 4% and 0.043 mIoU higher than EfficientNet-B0, and 7.4% and 0.006 mIoU higher than ConvNextV2-Atto pretrained on ImageNet, respectively.

Weed mapping in multispectral drone imagery using lightweight vision transformers

In precision agriculture, non-invasive remote sensing can be used to observe crops and weeds in visible and non-visible spectra. This paper proposes a novel approach for weed mapping using lightweight Vision Transformers. The method uses a lightweight Transformer architecture to process high-resolution aerial images obtained from drones and performs semantic segmentation to distinguish between crops and weeds. The method also employs specific architectural designs to enable transfer learning from RGB weights in a multispectral setting. For this purpose, the WeedMap dataset, acquired by drones equipped with multispectral cameras, was used. The experimental results demonstrate the effectiveness of the proposed method, exceeding the state-of-the-art. Our approach also enables more efficient mapping, allowing farmers to quickly and easily identify infested areas and prioritize their control efforts. These results encourage using drones as versatile computer vision flying devices for herbicide management, thereby improving crop yields. The code is available at https://github.com/pasqualedem/LWViTs-for-weedmapping.