Image Pyramid Research Articles

ENSEMBLE METHOD BASED ON AVERAGING SHAPES OF OBJECTS USING THE PYRAMID METHOD

Context. Image segmentation plays a key role in computer vision. The quality of segmentation is affected by many factors: noise, artifacts, complex shapes of objects. Classical methods cannot always guarantee good success, depending on the quality of the image and the existing noise, they cannot always achieve the desired result. The proposed method uses an ensemble of neural networks, which makes it possible to increase the accuracy and stability of segmentation. Objective. The goal of the work is to develop a new method of combining predictions of neural network ensembles, which can improve segmentation accuracy by combining images of different image sizes. Method. A method is proposed that averages the shapes of objects depicted on prediction masks. A pyramid of images is used to improve segmentation quality, each level of the pyramid corresponds to an increased size of the original image. This approach allows obtaining image characteristics at different levels. For a test image, a prediction is obtained from each neural network in the ensemble, after which a pyramid is built for the image. All pyramid levels are combined into the final image using SAAMC. All obtained final images for each neural network are also combined at the end using SAAMC. The use of an ensemble of neural networks combined with the pyramid method allows for reducing the impact of noise and artifacts on the segmentation results. Results. The use of this method was compared with the usual use of individual neural networks and the ensemble averaging method. The obtained results show that the proposed method outperforms its competitors. Application of the proposed method improved the accuracy and quality of segmentation. Conclusions. The conducted research confirmed the sense of using an ensemble of neural networks and creating a new method of combining predictions. The use of an ensemble of neural networks makes it possible to compensate for the errors and shortcomings of individual neural networks. Using the proposed method can significantly reduce the impact of noise and artifacts on segmentation. Further study and modification of this method will make it possible to further improve the quality of segmentation.

Learning to Holistically Detect Bridges From Large-Size VHR Remote Sensing Imagery.

Bridge detection in remote sensing images (RSIs) plays a crucial role in various applications, but it poses unique challenges compared to the detection of other objects. In RSIs, bridges exhibit considerable variations in terms of their spatial scales and aspect ratios. Therefore, to ensure the visibility and integrity of bridges, it is essential to perform holistic bridge detection in large-size very-high-resolution (VHR) RSIs. However, the lack of datasets with large-size VHR RSIs limits the deep learning algorithms' performance on bridge detection. Due to the limitation of GPU memory in tackling large-size images, deep learning-based object detection methods commonly adopt the cropping strategy, which inevitably results in label fragmentation and discontinuous prediction. To ameliorate the scarcity of datasets, this paper proposes a large-scale dataset named GLH-Bridge comprising 6,000 VHR RSIs sampled from diverse geographic locations across the globe. These images encompass a wide range of sizes, varying from 2,048 × 2,048 to 16,384 × 16,384 pixels, and collectively feature 59,737 bridges. These bridges span diverse backgrounds, and each of them has been manually annotated, using both an oriented bounding box (OBB) and a horizontal bounding box (HBB). Furthermore, we present an efficient network for holistic bridge detection (HBD-Net) in large-size RSIs. The HBD-Net presents a separate detector-based feature fusion (SDFF) architecture and is optimized via a shape-sensitive sample re-weighting (SSRW) strategy. The SDFF architecture performs inter-layer feature fusion (IFF) to incorporate multi-scale context in the dynamic image pyramid (DIP) of the large-size image, and the SSRW strategy is employed to ensure an equitable balance in the regression weight of bridges with various aspect ratios. Based on the proposed GLH-Bridge dataset, we establish a bridge detection benchmark including the OBB and HBB tasks, and validate the effectiveness of the proposed HBD-Net. Additionally, cross-dataset generalization experiments on two publicly available datasets illustrate the strong generalization capability of the GLH-Bridge dataset.

A Similar Feature Point Matching Method for Aerial Electric Power Tower Images Based on a One-Ring Neighborhood of Vertices

Due to the susceptibility of images to various factors such as scale changes, imaging conditions, and image noise, traditional feature point matching methods are difficult to achieve ideal matching accuracy, leading to many challenges in the automatic analysis and processing of power tower images. To improve the matching accuracy of similar feature points in aerial images of electric power tower, this study proposes a method for matching similar feature points in aerial images of electric power tower based on a vertex ring neighborhood, addressing the problem of large matching errors caused by factors such as scale and imaging conditions. This method first adopts the feature point extraction technique of electric power tower image based on decomposition and filtering, and constructs the Laplacian pyramid of the original image. Subsequently, filter banks are used to decompose the pyramid image in different directions, extract local extremum points as candidate feature point sets, and merge them to obtain the final feature point set. On this basis, taking into account the spatial relationships and geometric characteristics around the feature points, a texture mapping algorithm based on vertex ring neighborhood and patch classification is applied to construct a vertex ring neighborhood structure and extract geometrically representative electric power tower features. Finally, by using a texture feature point matching method based on the principle of similarity, the similarity between the real-time image and the reference image features is calculated for matching, and combined with the Babbitt coefficient to remove mismatched feature points, accurate matching of similar feature points in aerial electric power tower images is achieved. The experimental results show that this method has a mean square error of less than 0.1Pixel in matching similar texture feature points of aerial power tower images under various working conditions, significantly improving the matching accuracy and providing an effective tool for automatic analysis and processing of power tower images.

Integration of multi-modal datasets to estimate human aging

Aging involves complex biological processes leading to the decline of living organisms. As population lifespan increases worldwide, the importance of identifying factors underlying healthy aging has become critical. Integration of multi-modal datasets is a powerful approach for the analysis of complex biological systems, with the potential to uncover novel aging biomarkers. In this study, we leveraged publicly available epigenomic, transcriptomic and telomere length data along with histological images from the Genotype-Tissue Expression project to build tissue-specific regression models for age prediction. Using data from two tissues, lung and ovary, we aimed to compare model performance across data modalities, as well as to assess the improvement resulting from integrating multiple data types. Our results demostrate that methylation outperformed the other data modalities, with a mean absolute error of 3.36 and 4.36 in the test sets for lung and ovary, respectively. These models achieved lower error rates when compared with established state-of-the-art tissue-agnostic methylation models, emphasizing the importance of a tissue-specific approach. Additionally, this work has shown how the application of Hierarchical Image Pyramid Transformers for feature extraction significantly enhances age modeling using histological images. Finally, we evaluated the benefits of integrating multiple data modalities into a single model. Combining methylation data with other data modalities only marginally improved performance likely due to the limited number of available samples. Combining gene expression with histological features yielded more accurate age predictions compared with the individual performance of these data types. Given these results, this study shows how machine learning applications can be extended to/in multi-modal aging research. Code used is available at https://github.com/zroger49/multi_modal_age_prediction.

Computer-aided diagnosis system for grading brain tumor using histopathology images based on color and texture features

BackgroundCancer pathology shows disease development and associated molecular features. It provides extensive phenotypic information that is cancer-predictive and has potential implications for planning treatment. Based on the exceptional performance of computational approaches in the field of digital pathogenic, the use of rich phenotypic information in digital pathology images has enabled us to identify low-level gliomas (LGG) from high-grade gliomas (HGG). Because the differences between the textures are so slight, utilizing just one feature or a small number of features produces poor categorization results.MethodsIn this work, multiple feature extraction methods that can extract distinct features from the texture of histopathology image data are used to compare the classification outcomes. The successful feature extraction algorithms GLCM, LBP, multi-LBGLCM, GLRLM, color moment features, and RSHD have been chosen in this paper. LBP and GLCM algorithms are combined to create LBGLCM. The LBGLCM feature extraction approach is extended in this study to multiple scales using an image pyramid, which is defined by sampling the image both in space and scale. The preprocessing stage is first used to enhance the contrast of the images and remove noise and illumination effects. The feature extraction stage is then carried out to extract several important features (texture and color) from histopathology images. Third, the feature fusion and reduction step is put into practice to decrease the number of features that are processed, reducing the computation time of the suggested system. The classification stage is created at the end to categorize various brain cancer grades. We performed our analysis on the 821 whole-slide pathology images from glioma patients in the Cancer Genome Atlas (TCGA) dataset. Two types of brain cancer are included in the dataset: GBM and LGG (grades II and III). 506 GBM images and 315 LGG images are included in our analysis, guaranteeing representation of various tumor grades and histopathological features.ResultsThe fusion of textural and color characteristics was validated in the glioma patients using the 10-fold cross-validation technique with an accuracy equals to 95.8%, sensitivity equals to 96.4%, DSC equals to 96.7%, and specificity equals to 97.1%. The combination of the color and texture characteristics produced significantly better accuracy, which supported their synergistic significance in the predictive model. The result indicates that the textural characteristics can be an objective, accurate, and comprehensive glioma prediction when paired with conventional imagery.ConclusionThe results outperform current approaches for identifying LGG from HGG and provide competitive performance in classifying four categories of glioma in the literature. The proposed model can help stratify patients in clinical studies, choose patients for targeted therapy, and customize specific treatment schedules.

EEMSNet: Eagle-Eye Multi-Scale Supervised Network for cardiac segmentation

Cardiac segmentation plays a crucial role in computer-aided diagnosis of cardiac diseases. Nevertheless, the task of cardiac segmentation is inherently arduous due to the complex cardiac anatomy, and the intensity heterogeneity introduced during the acquisition of magnetic resonance images. To address these challenges, this paper presents an Eagle-Eye Multi-Scale Supervision Network. The network improves on the Full Convolutional Transformer network and introduces an innovative Eagle-Eye Multi-Scale Supervised module, which is used exclusively for edge supervision during the training process. The edge-supervised branch supervises the structure of cardiac edges at different scales within the input image pyramid structure. During training, the Eagle-Eye Multi-Scale Supervised module is based on multi-scale inputs. It uses edge information from Ground Truth to guide the model in learning the structural and detailed features of cardiac edges, ensuring the model captures edges accurately. Furthermore, considering that most cardiac segmentation networks typically overlook boundary profile information, our designed loss function places particular emphasis on the preservation of boundary profiles after the fusion of multi-scale edge features. Experimental results demonstrate that our proposed the Eagle-Eye Multi-Scale Supervision Network exhibits outstanding performance on the Automatic Cardiac Diagnosis Challenge dataset, achieving a notable average Dice Similarity Coefficient of 93.21%. Particularly, the Dice Similarity Coefficient value for right ventricle segmentation stands out at an impressive 93.52%. These findings signify the exceptional accuracy and effectiveness of our approach in addressing cardiac image segmentation tasks.

An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms.

Aiming at the problems of the poor robustness and universality of traditional contour matching algorithms in engineering applications, a method for improving the surface defect detection of industrial products based on contour matching algorithms is detailed in this paper. Based on the image pyramid optimization method, a three-level matching method is designed, which can quickly obtain the candidate pose of the target contour at the top of the image pyramid, combining the integral graph and the integration graph acceleration strategy based on weak classification. It can quickly obtain the rough positioning and rough angle of the target contour, which greatly improves the performance of the algorithm. In addition, to solve the problem that a large number of duplicate candidate points will be generated when the target candidate points are expanded, a method to obtain the optimal candidate points in the neighborhood of the target candidate points is designed, which can guarantee the matching accuracy and greatly reduce the calculation amount. In order to verify the effectiveness of the algorithm, functional test experiments were designed for template building function and contour matching function, including uniform illumination condition, nonlinear condition and contour matching detection under different conditions. The results show that: (1) Under uniform illumination conditions, the detection accuracy can be maintained at about 93%. (2) Under nonlinear illumination conditions, the detection accuracy can be maintained at about 91.84%. (3) When there is an external interference source, there will be a false detection or no detection, and the overall defect detection rate remains above 94%. It is verified that the proposed method can meet the application requirements of common defect detection, and has good robustness and meets the expected functional requirements of the algorithm, providing a strong technical guarantee and data support for the design of embedded image sensors in the later stage.

TRIPS: Trilinear Point Splatting for Real‐Time Radiance Field Rendering

AbstractPoint‐based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [KKLD23] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [RFS22] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen‐space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole‐free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions.Our evaluation demonstrate that TRIPS surpasses existing state‐of‐the‐art methods in terms of rendering quality while maintaining a real‐time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto‐exposed footage. The project page is located at:https://lfranke.github.io/trips

Online path planning of pork cutting robot using 3D laser point cloud

PurposeThe purpose of this paper is to propose an new online path planning method for porcine belly cutting. With the proliferation in demand for the automatic systems of pork production, the development of efficient and robust meat cutting algorithms are hot issues. The uncertain and dynamic nature of the online porcine belly cutting imposes a challenge for the robot to identify and cut efficiently and accurately. Based on the above challenges, an online porcine belly cutting method using 3D laser point cloud is proposed.Design/methodology/approachThe robotic cutting system is composed of an industrial robotic manipulator, customized tools, a laser sensor and a PC.FindingsAnalysis of experimental results shows that by comparing with machine vision, laser sensor-based robot cutting has more advantages, and it can handle different carcass sizes.Originality/valueAn image pyramid method is used for dimensionality reduction of the 3D laser point cloud. From a detailed analysis of the outward and inward cutting errors, the outward cutting error is the limiting condition for reducing the segments by segmentation algorithm.

User Engagement Detection-Based Financial Technology Advertising Video Effectiveness Evaluation

With the rapid advancement of financial technology, an increasing number of related advertisements have received widespread attention. User engagement detection during the advertisement viewing process directly reflects the effectiveness of the advertising video. Therefore, detecting user engagement during the advertisement viewing process has become a crucial issue. However, traditional engagement detection methods often require significant computational resources, significantly reducing their practicality. To address this issue, the authors propose a method to effectively detect user engagement by fully integrating multiple relatively practical models. Specifically, the authors extract key frame images from user face video and perform super-resolution reconstruction of them. Then image pyramid matching is used to achieve user engagement detection. Finally, the authors establish a reasonable database and conduct sufficient experiments based on it. Experimental results demonstrate that this proposed method has realistic engagement detection accuracy, and the design of multiple steps is also valid.

A real-time approximation algorithm of nonlinear filtering based on image pyramid

In the field of real-time rendering, post-processing filters are crucial techniques for achieving global illumination and photorealistic rendering in real-time rasterization and ray tracing rendering pipelines. Due to the power consumption limitations and insufficient memory bandwidth of mobile devices, filters can cause significant performance bottlenecks. To reduce the cost of filters on mobile devices, this paper proposes a real-time approximation method for nonlinear filtering. Our method reduce the number of texture samplings and memory usage. Through experiments conducted on mobile devices, it has been demonstrated that the proposed method can effectively reduce the number of texture samplings in feature images within scenes of similar complexity, as compared to mainstream industrial methods. This reduction in texture samplings leads to improved rendering quality while maintaining the same frame rate. Our method eliminates performance bottlenecks and enhances the quality of global illumination on mobile devices. By reducing the computational complexity of the non-linear filter, it provides users with a smoother experience.

Rethinking Multi-Scale Representations in Deep Deraining Transformer

Existing Transformer-based image deraining methods depend mostly on fixed single-input single-output U-Net architecture. In fact, this not only neglects the potentially explicit information from multiple image scales, but also lacks the capability of exploring the complementary implicit information across different scales. In this work, we rethink the multi-scale representations and design an effective multi-input multi-output framework that constructs intra- and inter-scale hierarchical modulation to better facilitate rain removal and help image restoration. We observe that rain levels reduce dramatically in coarser image scales, thus proposing to restore rain-free results from the coarsest scale to the finest scale in image pyramid inputs, which also alleviates the difficulty of model learning. Specifically, we integrate a sparsity-compensated Transformer block and a frequency-enhanced convolutional block into a coupled representation module, in order to jointly learn the intra-scale content-aware features. To facilitate representations learned at different scales to communicate with each other, we leverage a gated fusion module to adaptively aggregate the inter-scale spatial-aware features, which are rich in correlated information of rain appearances, leading to high-quality results. Extensive experiments demonstrate that our model achieves consistent gains on five benchmarks.

LaMAR: Laplacian Pyramid for Multimodal Adaptive Super Resolution (Student Abstract)

Recent advances in image-to-image translation involve the integration of non-visual imagery in deep models. Non-visual sensors, although more costly, often produce low-resolution images. To combat this, methods using RGB images to enhance the resolution of these modalities have been introduced. Fusing these modalities to achieve high-resolution results demands models with millions of parameters and extended inference times. We present LaMAR, a lightweight model. It employs Laplacian image pyramids combined with a low-resolution thermal image for Guided Thermal Super Resolution. By decomposing the RGB image into a Laplacian pyramid, LaMAR preserves image details and avoids high-resolution feature map computations, ensuring efficiency. With faster inference times and fewer parameters, our model demonstrates state-of-the-art results.

Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

Artistic image completion assumes a significant role in the preservation and restoration of invaluable art paintings, marking notable advancements through the adoption of deep learning methodologies. Despite progress, challenges persist, particularly in achieving optimal results for high-resolution paintings. The intricacies of complex structures and textures in art paintings pose difficulties for sophisticated approaches like Generative Adversarial Networks (GANs), leading to issues such as small-scale texture synthesis and the inference of missing information, resulting in distortions in lines and unnatural colors. Simultaneously, patch-based image synthesis, augmented with global optimization on the image pyramid, has evolved to enhance structural coherence and details. However, gradient-based synthesis methods face obstacles related to directionality, inconsistency, and the computational burdens associated with solving the Poisson equation in non-integrable gradient fields. This paper introduces a pioneering approach, integrating Weighted Similarity-Confidence Laplacian Synthesis to comprehensively address these challenges and advance the field of artistic image completion. Experimental results affirm the effectiveness of our approach, offering promising outcomes for the preservation and restoration of art paintings with intricate details and irregular missing regions. The integration of weighted Laplacian synthesis and patch-based completion across multi-regions ensures precise and targeted completion, outperforming existing methods. A comparative analysis underscores our method’s superiority in artifact reduction and minimizing blurriness, particularly addressing challenges related to color discrepancies in texture areas. Additionally, the incorporation of pyramid blending proves advantageous, ensuring smoother transitions and preventing noticeable seams or artifacts in blended results. Based on empirical results, our method consistently outperforms previous methods across both high and low resolutions. Responding to these insights, our approach emerges as an invaluable guide for both curators and artists. The algorithm’s performance yields insights that underscore the central role of thoughtful decision making in the creation of art paintings. This guidance extends to informing choices related to color selection, brushstrokes, and various other elements integral to the artistic process. During the creation phase, employing these insights enables artists and curators to optimize not only the digitization but also the subsequent restoration process. This proves especially vital when dealing with the intricacies involved in physically restoring damaged original art paintings. Importantly, our approach not only streamlines the restoration process but also contributes significantly to the preservation and enhancement of the digital representations of these distinctive and often intricate works of art.

EAP: An effective black-box impersonation adversarial patch attack method on face recognition in the physical world

Face recognition models and systems based on deep neural networks are vulnerable to adversarial examples. However, existing attacks on face recognition are either impractical or ineffective for black-box impersonation attacks in the physical world. In this paper, we propose EAP, an effective black-box impersonation attack method on face recognition in the physical world. EAP generates adversarial patches that can be printed by mobile and compact printers and attached to the source face to fool face recognition models and systems. To improve the transferability of adversarial patches, our approach incorporates random similarity transformations and image pyramid strategies, increasing input diversity. Furthermore, we introduce a meta-ensemble attack strategy that harnesses multiple pre-trained face models to extract common gradient features. We evaluate the effectiveness of EAP on two face datasets, using 16 state-of-the-art face recognition backbones, 9 heads, and 5 commercial systems. Moreover, we conduct physical experiments to substantiate its practicality. Our results demonstrate that EAP is capable of effectively executing impersonation attacks against state-of-the-art face recognition models and systems in both digital and physical environments.

Independently Trained Multi-Scale Registration Network Based on Image Pyramid.

Image registration is a fundamental task in various applications of medical image analysis and plays a crucial role in auxiliary diagnosis, treatment, and surgical navigation. However, cardiac image registration is challenging due to the large non-rigid deformation of the heart and the complex anatomical structure. To address this challenge, this paper proposes an independently trained multi-scale registration network based on an image pyramid. By down-sampling the original input image multiple times, we can construct image pyramid pairs, and design a multi-scale registration network using image pyramid pairs of different resolutions as the training set. Using image pairs of different resolutions, train each registration network independently to extract image features from the image pairs at different resolutions. During the testing stage, the large deformation registration is decomposed into a multi-scale registration process. The deformation fields of different resolutions are fused by a step-by-step deformation method, thereby addressing the challenge of directly handling large deformations. Experiments were conducted on the open cardiac dataset ACDC (Automated Cardiac Diagnosis Challenge); the proposed method achieved an average Dice score of 0.828 in the experimental results. Through comparative experiments, it has been demonstrated that the proposed method effectively addressed the challenge of heart image registration and achieved superior registration results for cardiac images.

Improving noise characteristics using a modified image pyramid with guided filtering and Bayesian shrinkage threshold in low-dose animal radiography

Radiography is a commonly used diagnostic tool in animal practice to provide clinical information. The recent market for animal radiography is rapidly growing owing to the increasing awareness of the welfare of companion animals and the expansion of animal clinics. As the clinical use of radiography in animal hospitals has grown rapidly, radiologists continuously seek ways to reduce the radiation dose to animals. This study proposes an effective noise removal method using a modified image pyramid with guided filtering and Bayesian shrinkage threshold for low-dose animal radiography. This study aims to model a modified image pyramid-based denoising algorithm and to confirm its applicability in an animal radiographic system. We conducted an experiment on a companion dog and cat using a commercially-available veterinary radiographic system and quantitatively evaluated the image quality using the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The PSNR and SSIM values measured in the denoised image of the dog using the proposed denoising algorithm were 39.54 and 0.96, respectively, which are improvements of approximately 11.3% and 5.5% of the values obtained in the original noisy image. Consequently, the proposed method is highly efficient in reducing image noise in animal radiography, thereby improving the image quality.

Deformable 3D/3D CT-to-digital-tomosynthesis image registration in image-guided bronchoscopy interventions

Traditional navigational bronchoscopy procedures rely on preprocedural computed tomography (CT) and intraoperative chest radiography and cone-beam CT (CBCT) to biopsy peripheral lung lesions. This navigational approach is challenging due to the projective nature of radiography, and the high radiation dose, long imaging time, and large footprints of CBCT. Digital tomosynthesis (DTS) is considered an attractive alternative combining the advantages of radiography and CBCT. Only the depth resolution cannot match a full CBCT image due to the limited angle acquisition. To address this issue, preoperative CT is a good auxiliary in guiding bronchoscopy interventions. Nevertheless, CT-to-body divergence caused by anatomic changes and respiratory motion, hinders the effective use of CT imaging. To mitigate CT-to-body divergence, we propose a novel deformable 3D/3D CT-to-DTS registration algorithm employing a multistage, multiresolution approach and using affine and elastic B-spline transformation models with bone and lung mask images. A multiresolution strategy with a Gaussian image pyramid and a multigrid strategy within the B-spline model are applied. The normalized correlation coefficient is included in the cost function for the affine model and a multimetric weighted cost function is used for the B-spline model, with weights determined heuristically. Tested on simulated and real patient bronchoscopy data, the algorithm yields promising results. Assessed qualitatively by visual inspection and quantitatively by computing the Dice coefficient (DC) and the average symmetric surface distance (ASSD), the algorithm achieves mean DC of 0.82±0.05 and 0.74±0.05, and mean ASSD of 0.65±0.29mm and 0.93±0.43mm for simulated and real data, respectively. This algorithm lays the groundwork for CT-aided intraoperative DTS imaging in image-guided bronchoscopy interventions with future studies focusing on automated metric weight setting.

Integrating prior knowledge into a bibranch pyramid network for medical image segmentation

Medical image segmentation is crucial for obtaining accurate diagnoses, and while convolutional neural network (CNN)-based methods have made strides in recent years, they struggle with modeling long-range dependencies. Transformer-based methods improve this task but require more computational resources. The segment anything model (SAM) can generate pixel-level segmentation results for natural images using sparse manual prompts, but it performs poorly on low-contrast, noisy ultrasound images. To address this issue, we propose a new medical image segmentation network architecture that integrates transformer components, CNN modules, and an SAM encoder into a unified framework. This allows us to simultaneously capture both long-range dependencies and local features. Additionally, we incorporate the image features extracted from the SAM model as prior knowledge to achieve further improved segmentation accuracy with limited training data. To reduce the imposed computational stress, we employ an axial attention mechanism to approximate a transformer's effects by expanding the receptive field. Instead of replacing the transformer components with lightweight attention modules, our model is divided into a global branch and a local branch. The global branch extracts context features with the transformer components, while the local branch processes patch tokens with the axial attention mechanism. We also construct an image pyramid to excavate internal statistics and multiscale representations to obtain more accurate segmentation regions. This bibranch pyramid transformer (Bi-BPT) architecture is effective and robust for medical image segmentation, surpassing other related segmentation network architectures. The experimental results obtained on various medical image datasets demonstrate its effectiveness.

Underwater dynamic polarization imaging without dependence on the background region.

Active-polarization imaging holds significant promise for achieving clear underwater vision. However, only static targets were considered in previous studies, and a background region was required for image restoration. To address these issues, this study proposes an underwater dynamic polarization imaging method based on image pyramid decomposition and reconstruction. During the decomposition process, the polarized image is downsampled to generate an image pyramid. Subsequently, the spatial distribution of the polarization characteristics of the backscattered light is reconstructed by upsampling, which recovered the clear scene. The proposed method avoids dependence on the background region and is suitable for moving targets with varying polarization properties. The experimental results demonstrate effective elimination of backscattered light while sufficiently preserving the target details. In particular, for dynamic targets, processing times that fulfill practical requirements and yield superior recovery effects are simultaneously obtained.