Region Adjacency Graph Research Articles

SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images.

Blood vessel and surgical instrument segmentation is a fundamental technique for robot-assisted surgical navigation. Despite the significant progress in natural image segmentation, surgical image-based vessel and instrument segmentation are rarely studied. In this work, we propose a novel self-supervised pretraining method (SurgNet) that can effectively learn representative vessel and instrument features from unlabeled surgical images. As a result, it allows for precise and efficient segmentation of vessels and instruments with only a small amount of labeled data. Specifically, we first construct a region adjacency graph (RAG) based on local semantic consistency in unlabeled surgical images and use it as a self-supervision signal for pseudo-mask segmentation. We then use the pseudo-mask to perform guided masked image modeling (GMIM) to learn representations that integrate structural information of intraoperative objectives more effectively. Our pretrained model, paired with various segmentation methods, can be applied to perform vessel and instrument segmentation accurately using limited labeled data for fine-tuning. We build an Intraoperative Vessel and Instrument Segmentation (IVIS) dataset, comprised of ~3 million unlabeled images and over 4,000 labeled images with manual vessel and instrument annotations to evaluate the effectiveness of our self-supervised pretraining method. We also evaluated the generalizability of our method to similar tasks using two public datasets. The results demonstrate that our approach outperforms the current state-of-the-art (SOTA) self-supervised representation learning methods in various surgical image segmentation tasks.

Region adjacency graph based GNN approach for static signature classification

Graph Neural Networks (GNNs) have gained popularity across various research fields in recent years. GNNs utilize graphs to construct an embedding that includes details about the nodes and edges in a graph’s neighborhood. In this work, a set of Region Adjacency Graphs (RAG) derives the attribute values from Static Signature (SS) images. These attribute values are used to label the nodes of the complete graph, which is formed by considering each signature as a node taken from the sample of signatures of a specific signer. The complete graph is trained by using GraphSAGE, an inductive representation learning method. This trained model helps to determine any newly introduced node (static signature to be tested) as genuine or fake. Standard static signature datasets, notably GPDSsynthetic and MCYT-75 are used to test the prevailing model. Experimental results on genuine and counterfeit signature networks demonstrate that our computed model enables a high rate of accuracy (GPDSsynthetic 99.91% and MCYT-75 99.56%) and minimum range of loss (GPDSsynthetic 0.0061 and MCYT-75 0.0070) on node classification.

Automated detection and tracking of medium-large icebergs from Sentinel-1 imagery using Google Earth Engine

Monitoring Antarctic icebergs helps us understand the interaction between ocean, atmosphere, and sea ice in the Southern Ocean. Although gigantic icebergs have been the objects of many remote sensing studies, medium icebergs in the Southern Ocean have been rarely monitored or traced. In this study, we develop an iceberg detection and tracking tool particularly for medium and large icebergs (0.4–10 km2), by using Python programming language and Sentinel-1 (S1) imagery, based on Google Earth Engine (GEE). To detect icebergs, we employ the simple non-iterative clustering (SNIC) and region adjacency graph (RAG) merging for object-based image segmentation and train/test the support vector machine (SVM) model with 6432 labeled segments of iceberg or non-icebergs from 40 images of S1 (2019–2021). Radar backscatter features and morphological features of those segments are used as the inputs of the SVM model. After icebergs are detected in two image scenes of different dates, we track the displacements of detected icebergs by comparing their 1-D shape signals. Our model shows ∼99% of accuracy in detecting icebergs and ∼ 93–98% of accuracy in tracking icebergs depending on the day difference between image scenes. When using our tool for the monitoring of icebergs in the Amundsen Sea, we find that the iceberg fraction varies from 2% to 8% in 2021 and most of icebergs move westward with a speed of <0.2 km/day.

Building element recognition with MTL-AINet considering view perspectives

Abstract The reconstruction and analysis of building models are crucial for the construction of smart cities. A refined building model can provide a reliable data support for data analysis and intelligent management of smart cities. The colors, textures, and geometric forms of building elements, such as building outlines, doors, windows, roof skylights, roof ridges, and advertisements, are diverse; therefore, it is challenging to accurately identify the various details of buildings. This article proposes the Multi-Task Learning AINet method that considers features such as color, texture, direction, and roll angle for building element recognition. The AINet is used as the basis function; the semantic projection map of color and texture, and direction and roll angle is used for multi-task learning, and the complex building facade is divided into similar semantic patches. Thereafter, the multi-semantic features are combined using hierarchical clustering with a region adjacency graph and the nearest neighbor graph to achieve an accurate recognition of building elements. The experimental results show that the proposed method has a higher accuracy for building detailed edges and can accurately extract detailed elements.

Pruning for image segmentation: Improving computational efficiency for large-scale remote sensing applications

Image segmentation is a fundamental step in object-based image analysis and other workflows. However, high-efficiency remains a challenge, especially for the analysis of large-scale Earth observation images. In recent years, considerable effort has been paid to designing merging criteria, automatic scale selection, and object-specific optimisation. These segmentation methods usually rely on the region-adjacency graph (RAG) model and the nearest neighbour graph (NNG) model, which provide acceptable merging performance. Low efficiency occurs due to many redundant edge weight updates in the RAG model. In this study, we propose a generic dynamic pruning framework to improve the efficiency of existing region-merging-based segmentation algorithms, opening the door for large-scale applications in remote sensing. The proposed pruning framework includes intra-object and inter-object pruning modules for the RAG model. Inter-object pruning divides the RAG model into multiple sub-RAG models to reduce the redundant edge weight updates between adjacent objects. Intra-object pruning iteratively divides the sub-RAG into smaller RAGs. In our experimental analysis, we employ the proposed pruning framework with six region-merging segmentation methods and validate the effectiveness on four 10–20M pixel images and a 100M pixel data set. The pruning framework improves the performance of various segmentation algorithms by reducing computational complexity while maintaining segmentation accuracy. We observed a significant improvement in efficiency, with various achieving super-linear speed-up while maintaining the stability of segmentation accuracy. In single-core mode, the computation time of tested algorithms is enhanced by two to ten times on the four test images. In the multicore mode, speed-up increased up to 40 times with eight CPU cores. The computational cost was reduced by 36.15% to 95.77% in the number of weight updates, which is independent of hardware characteristics. On the large-scale image, two modes achieved speed-ups of 36.07 and 102.74, respectively.

Fully Parallel Homological Region Adjacency Graph via Frontier Recognition

Relating image contours and regions and their attributes according to connectivity based on incidence or adjacency is a crucial task in numerous applications in the fields of image processing, computer vision and pattern recognition. In this paper, the crucial incidence topological information of 2-dimensional images is extracted in an efficient manner through the computation of a new structure called the HomDuRAG of an image; that is, the dual graph of the HomRAG (a topologically consistent extended version of the classical RAG). These representations are derived from the two traditional self-dual square grids (in which physical pixels play the role of 2-dimensional cells) and encapsulate the whole set of topological features and relations between the three types of objects embedded in a digital image: 2-dimensional (regions), 1-dimensional (contours) and 0-dimensional objects (crosses). Here, a first version of a fully parallel algorithm to compute this new representation is presented, whose timing complexity order (in the worst case and supposing one processing element per 0-cell) is O(log(M×N)) , M and N being the height and width of the image. Efficient implementations of this parallel algorithm would allow images to be processed in real time, as well as permit us to uncover fast algorithms for contour detection and segmentation, opening new perspectives within the image processing field.

Detection of Massive Oil Spills in Sun Glint Optical Imagery through Super-Pixel Segmentation

Large volumes of crude oil accidentally released into the sea may cause irreversible adverse impacts on marine and coastal environments. Large swath optical imagery, acquired using platforms such as the moderate-resolution imaging spectroradiometer (MODIS), is frequently used for massive oil spill detection, attributing to its large coverage and short global revisit, providing rich data for oil spill monitoring. The aim of this study was to develop a suitable approach for massive oil spill detection in sun glint optical imagery. Specifically, preprocessing procedures were conducted to mitigate the inhomogeneous light field over the spilled area caused by sun glint, enhance the target boundary contrast, and maintain the internal homogeneity within the target. The image was then segmented into super-pixels based on a simple linear clustering method with similar characteristics of color, brightness, and texture. The neighborhood super-pixels were merged into target objects through the region adjacency graph method based on the Euclidean distance of their colors with an adaptive termination threshold. Oil slicks from the generated bright/dark objects were discriminated through a decision tree with parameters based on spectral and spatial characteristics. The proposed approach was applied to oil spill detection in MODIS images acquired during the Montara oil spill in 2009, with an overall extraction precision of 0.8, recall of 0.838, and F1-score of 0.818. Such an approach is expected to provide timely and accurate oil spill detection for disaster emergency response and ecological impact assessment.

China's oases have expanded by nearly 40% over the past 20 years

AbstractOases are supporters of biological, social, and economic activities in arid regions, which guides their prosperity or decay. Given that China has the largest oases area in the world, determining their evolution is of substantial importance for sustainable development and the ecological protection of these systems. Accordingly, the spatiotemporal evolution of China's oases between 2000 and 2020 was analyzed. This was based on the normalized difference vegetation index derived from moderate imaging spectroradiometer data, combined with superpixel image segmentation, the region adjacency graph merging method, and the genetic algorithm. The results were as follows. Between 2000 and 2020: (1) the oases in China expanded significantly from 207,812 to 289,751 km2; with a total increase of 39.43%; and the number of oases decreased from 135 to 93, which mainly occurred in micro and small oases; (2) the areas of oases in stable, expanding, and shrinking states accounted for 96.77%, 42.65%, and respectively 3.23% of the total oasis area in 2000. Oases have become more concentrated and continuously expanded; (3) the increase in glacial meltwater and precipitation caused by rising temperatures increased the availability of water resources in oases, which provides the substantial basis for expansion; and (4) the primary anthropogenic drivers of oasis expansion and stabilization regions were cultivation and construction, respectively.

CMSuG: Competitive mechanism-based superpixel generation method for image segmentation

During the last years, object-based image segmentation (OBIA) has seen a considerable increase in the image segmentation. OBIA is generally based on superpixel methods, in which the clustering-based method plays an increasingly important role. Most clustering methods for generating superpixels suffer from inaccurate classification points with inappropriate cluster centers. To solve the problem, we propose a competitive mechanism-based superpixel generation (CMSuG) method, which both accelerates convergence and promotes robustness for noise sensitivity. Then, image segmentation results will be obtained by a region adjacent graph (RAG)-based merging algorithm after constructing an RAG. However, high segmentation accuracy is customarily accompanied by expensive time-consuming costs. To improve computational efficiency, we address a parallel CMSuG algorithm, the time of which is much less than the CMSuG method. In addition, we present a parallel RAG method to decrease the expensive time-consuming cost in serial RAG construction. By leveraging parallel techniques, the running time of the whole image segmentation method decline with the time complexity from O (N) + O (K2) to O (N/K) or O (K2), in which N is the size of an input image and K is the given number of the superpixel. In the experiments, both nature image and remote sensing image segmentation results demonstrate that our CMSuG method outperforms the state-of-the-art superpixel generation methods, and then performs well for image segmentation in turn. Compared with the serial segmentation method, our parallel techniques gain more than four times acceleration in both remote sensing image dataset and nature image dataset.

GLeSI: A system for extraction of glacial lakes using satellite imagery

SummaryFor outburst susceptibility assessment of glacial lakes, manual field survey is a time‐consuming, expensive, and very tedious process. Therefore, an automatic and reliable system is required for efficient extraction of glacial lake's from satellite imagery. Low spectral contrast and heterogeneous backgrounds of satellite images are the major challenges for finding information of the glacial lake's region. Therefore, to overcome these challenges, an automatic technique for extraction of glacial lakes using satellite imagery which is termed as GLeSI, is proposed in this article. In the GLeSI system, normalized cut (Ncut) segmentation technique with region adjacency graph and simple linear iterative clustering is proposed for accurate extraction of the glacial lake's region. The proposed system achieved an overall accuracy of 93.21%, 91.57%, 97.65% on Landsat 8 datasets for Imja, Chandra basin, and Bhaga basin glacier lake region, respectively. The qualitative and quantitative performance analysis shows the significant improvement of the proposed technique compared to state‐of‐the‐art techniques for extraction of glacial lake's region.

A Method Combining Line Detection and Semantic Segmentation for Power Line Extraction from Unmanned Aerial Vehicle Images

Power line extraction is the basic task of power line inspection with unmanned aerial vehicle (UAV) images. However, due to the complex backgrounds and limited characteristics, power line extraction from images is a difficult problem. In this paper, we construct a power line data set using UAV images and classify the data according to the image clutter (IC). A method combining line detection and semantic segmentation is used. This method is divided into three steps: First, a multi-scale LSD is used to determine power line candidate regions. Then, based on the object-based Markov random field (OMRF), a weighted region adjacency graph (WRAG) is constructed using the distance and angle information of line segments to capture the complex interaction between objects, which is introduced into the Gibbs joint distribution of the label field. Meanwhile, the Gaussian mixture model is utilized to form the likelihood function by taking the spectral and texture features. Finally, a Kalman filter (KF) and the least-squares method are used to realize power line pixel tracking and fitting. Experiments are carried out on test images in the data set. Compared with common power line extraction methods, the proposed algorithm shows better performance on images with different IC. This study can provide help and guidance for power line inspection.

An unsupervised approach of colonic polyp segmentation using adaptive markov random fields

• Unsupervised segmentation of polyps in colonoscopic images. • Superpixel level grouping of pixels using simple linear iterative clustering (SLIC). • Deployment of adaptive Markov random field (MRF) to stitch homogeneous regions with region adjacency graph (RAG). • Polyp segmentation efficiency of 60.77% is achieved in terms of dice co-efficient. In this article, an unsupervised image segmentation method is proposed to segment out polyps in endoscopic images. Based on the skeleton of an over-segmented image, an adaptive Markov Random Field (MRF)-based framework is employed. The polyps or abnormal regions show markedly different texture and color characteristics in contrast to normal tissues. In our method, the endoscope images are first over-segmented into superpixels . For final refinement, Local binary pattern (LBP) and color features are used in the adaptive MRF. The experimental results show that the proposed method can perfectly localize polyp regions, and it can give a mean dice value of 60.77% in a test set of 612 images taken from 29 different video sequences. Deep learning models require a huge number of training images for tweaking of network parameters. In this context, it is sometimes challenging to deploy a deep model for unseen medical datasets having many segmentation challenges, like complex background, specularity, non-uniform illuminations, etc. Also, manual labeling of a huge number of images is another challenge. That is why we adopted an unsupervised approach in this paper.

Color Image Segmentation by Utilizing Coarse-to-Fine Strategy

Image segmentation is an important processing technology, which is the basis of image recognition and has been widely used in many fields. In this paper, we propose a method, termed coarse-to-fine strategy-based image segmentation (CSIS), for color image segmentation. The basic idea is to segment an image by three phases: (1) the original image is first segmented into several distinct regions by using the mean shift method; (2) the segmented regions are converted to a weighted region adjacency graph (RAG), and a new graph cut method, called multi-cut algorithm, is proposed to partition the RAG into multiple regions; (3) a one-step Chan–Vese algorithm is applied to smooth the boundaries of the segmented objectives. In each of the last two phases, a method is applied to refine the result obtained in the previous phase. By carefully balancing the efforts used in each phase, CSIS could segment color images both efficiently and effectively. These advantages are demonstrated by applying the proposed method to a variety of test instances, and the statistical results also show that it is comparable with some state-of-the-art methods.

Infrared Image Superpixel Segmentation Based on Seed Strategy of Contour Encoding

Superpixel segmentation has become a crucial pre-processing tool to reduce computation in many computer vision applications. In this paper, a superpixel extraction algorithm based on a seed strategy of contour encoding (SSCE) for infrared images is presented, which can generate superpixels with high boundary adherence and compactness. Specifically, SSCE can solve the problem of superpixels being unable to self-adapt to the image content. First, a contour encoding map is obtained by ray scanning the binary edge map, which ensures that each connected domain belongs to the same homogeneous region. Second, according to the seed sampling strategy, each seed point can be extracted from the contour encoding map. The initial seed set, which is adaptively scattered based on the local structure, is capable of improving the capability of boundary adherence, especially for small regions. Finally, the initial superpixels limited by the image contour are generated by clustering and refined by merging similar adjacent superpixels in the region adjacency graph (RAG) to reduce redundant superpixels. Experimental results on a self-built infrared dataset and the public datasets BSD500 and 3Dircadb demonstrate the generalization ability in grayscale and medical images, and the superiority of the proposed method over several state-of-the-art methods in terms of accuracy and compactness.

Attention-Driven Graph Convolution Network for Remote Sensing Image Retrieval

Graph convolution networks (GCNs) are useful in remote sensing (RS) image retrieval. It is found to be effective because, in a graph representation, the relative geometrical interactions between different regions (or segments) are appropriately captured, along with their region-wise features in their region adjacency graphs. Also, the attention mechanism has often been applied to the nodes to highlight the essential features in each node. In this regard, a significant amount of high-frequency information is missed since each image segment is effectively summarized within a single node. To account for this and increase the learning capacity, we propose to attend over the edge/adjacency matrix to highlight the interactions among meaningful regions that contribute to supervised learning from images. We exploit this novel edge attention mechanism together with node attention to highlight essential image context by allowing more importance to the meaningful neighboring regions that highlight a relevant node. We implement the proposed context-attended GCN framework for image retrieval on the benchmarked UC-Merced and the PatternNet datasets. We observe a notable improvement in the results compared to the state of the art.

Graph Convolutional Networks for Semi-Supervised Image Segmentation

The problem of image segmentation is one of the most significant ones in computer vision. Recently, deep-learning methods have dominated state-of-the-art solutions that automatically or interactively divide an image into subregions. However, the limitation of deep-learning approaches is that they require a substantial amount of training data, which is costly to prepare. An alternative solution is semi-supervised image segmentation. It requires rough denotations to define constraints that are next generalized to precisely delimit relevant image regions without using train examples. Among semi-supervised strategies for image segmentation, the leading are graph-based techniques that define image segmentation as a result of pixel or region affinity graph partitioning. This paper revisits the problem of graph-based image segmentation. It approaches the problem as semi-supervised node classification in the SLIC superpixels region adjacency graph using a graph convolutional network (GCN). The performance of both spectral and spatial graph convolution operators is considered, represented by Chebyshev convolution operator and GraphSAGE respectively. The results of the proposed method applied to binary and multi-label segmentation are presented, numerically assessed, and analyzed. In its best variant, the proposed method scored the average DICE of 0.86 in the binary segmentation task and 0.79 in the multi-label segmentation task. Comparison with state-of-the-art graph-based methods, including Random Walker and GrabCut, shows that graph convolutional networks can represent an attractive alternative to the existing solutions to graph-based semi-supervised image segmentation.

Symbols Detection and Classification using Graph Neural Networks

In this paper, we propose a method to both extract and classify symbols in floorplan images. This method relies on the very recent developments of Graph Neural Networks (GNN). In the proposed approach, floorplan images are first converted into Region Adjacency Graphs (RAGs). In order to achieve both classification and extraction, two different GNNs are used. The first one aims at classifying each node of the graph while the second targets the extraction of clusters corresponding to symbols. In both cases, the model is able to take into account edge features. Each model is firstly evaluated independently before combining both tasks simultaneously, increasing the quickness of the results.

Automatic Segmentation of Digital Pathology Slides Based on Unsupervised Learning

To segment images through an unsupervised method as an alternative to manual labeling. A total of 100 whole slide image (WSI) data of HE stained and Pap stained slides were selected as the research and test objects, including 70 breast slides, 20 lung slides and 10 thyroid slides. In order to ensure the diversity of data, the breast slides included those of normal tissue, inflammation and tumor, the lung slides were mainly neoplasms in the lower lobe, including those of inflammation and tumor, and the thyroid slides were of cells, all benign, obtained through fine needle aspiration. The maximum total magnification (original magnification) of each image was 400 times, and the file format was NDPI. Each WSI was manually labeled, and the labeled area of each WSI was more than 10 fields of vision. The labeled information was to be used for validity verification. An unsupervised image segmentation technique based on superpixel and fully convolution neural network algorithms was constructed and used to segment any region of interest (ROI) of unlabeled WSI. In comparison with the region adjacency graph merging method, the segmentation effect of the two methods was assessed with the under segmentation error, the boundary recall and the mean Intersection-over-Union, and the efficiency of the two methods was also compared. In the comparison of execution efficiency, the test process included the preprocessing time of superpixel, and excluded the time of loading the deep learning engine. Unsupervised automatic segmentation was implemented for any ROI region of WSI according to the texture and color. The results of the breast slides, lung slides and thyroid slides showed slight differences, and multiple tests yielded stable results. However, the performance of this method in differentiating inflammation and tumor was average. The under-segmentation error, the boundary recall and the mean Intersection-over-Union were 19.10%, 82.06% and 45.06%, respectively. The under segmentation error, the boundary recall and the mean Intersection-over-Union for the region adjacency graph merging method were 21.52%, 78.39% and 44.81%, respectively. The average time consumption of the whole process was 0.27 s in GPU mode and 1.30 s in CPU mode. The average time consumption of the region adjacency graph merging method was 10.5 s in CPU mode because the method of region adjacency graph merging was not realized in the GPU mode. This method produced ideal pixel level labeling results through simple human-computer interaction, which could effectively reduce the cost of digital pathology slide data labeling. Compared with the region adjacency graph merging method, this method had better performance in processing image texture and had faster processing speed.

An unsupervised method for histological image segmentation based on tissue cluster level graph cut

While deep learning models have demonstrated outstanding performance in medical image segmentation tasks, histological annotations for training deep learning models are usually challenging to obtain, due to the effort and experience required to carefully delineate tissue structures. In this study, we propose an unsupervised method, termed as tissue cluster level graph cut (TisCut), for segmenting histological images into meaningful compartments (e.g., tumor or non-tumor regions), which aims at assisting histological annotations for downstream supervised models. The TisCut consists of three modules. First, histological tissue objects are clustered based on their spatial proximity and morphological features. The Voronoi diagram is then constructed based on tissue object clustering. In the last module, morphological features computed from the Voronoi diagram are integrated into a region adjacency graph. Image partition is then performed to divide the image into meaningful compartments by using the graph cut algorithm. The TisCut has been evaluated on three histological image sets for necrosis and melanoma detections. Experiments show that the TisCut could provide a comparative performance with U-Net models, which achieves about 70% and 85% Jaccard index coefficients in partitioning brain and skin histological images, respectively. In addition, it shows the potential to be used for generating histological annotations when training masks are difficult to collect for supervised segmentation models.

A Graph Neural Network for superpixel image classification

The classification of superpixel images by graph neural networks has gradually become a research hotspot. It is a crucial issue to embed super-pixel images from lowdimensional to high-dimensional so as to turn complex image information into graph signals. This paper proposes a method for image classification using a graph neural network (GNN) model. We convert the input image into a region adjacency graph (RAG) composed of superpixels as nodes, and use residual and concat structure to extract deep features. Finally, the loss function that increases the distance between classes and compactness within classes is used as supervision. Experiments have been tested with different numbers of superpixels on multiple datasets, and the results show that our method has a great performance in superpixel images classification.