Group Of Pixels Research Articles

GOSS: towards generalized open-set semantic segmentation

In this paper, we extend Open-set Semantic Segmentation (OSS) into a new image segmentation task called Generalized Open-set Semantic Segmentation (GOSS). Previously, with well-known OSS, the intelligent agents only detect unknown regions without further processing, limiting their perception capacity of the environment. It stands to reason that further analysis of the detected unknown pixels would be beneficial for agents’ decision-making. Therefore, we propose GOSS, which holistically unifies the abilities of two well-defined segmentation tasks, i.e. OSS and generic segmentation. Specifically, GOSS classifies pixels as belonging to known classes, and clusters (or groups) of pixels of unknown class are labelled as such. We propose a metric that balances the pixel classification and clustering aspects to evaluate this newly expanded task. Moreover, we build benchmark tests on existing datasets and propose neural architectures as baselines. Our experiments on multiple benchmarks demonstrate the effectiveness of our baselines. Code is made available at https://github.com/JHome1/GOSS_Segmentor.

Intra-hour photovoltaic forecasting through a time-varying Markov switching model

This work presents a Markov switching model with a time-varying transition matrix to forecast intra-hour photovoltaic (PV) power output, aiming at providing forecasting flexibility. First, the proposed methodology captures images of the sky employing a self-made, low-cost all-sky imager. Second, the goal is to limit exposure problems in those images via the exposure fusion technique. Third, the proposed algorithm identifies groups of pixels forming clouds through a super paramagnetic clustering algorithm. Finally, we model the problem with a homogeneous Poisson process and forecast the cloud location and the shadowed area on a PV plant for the coming minutes. The shadowed area together with meteorological data are the inputs to this model. In the case study, our approach shows better performance than the persistence method, in particular for changing cloud conditions.

Delineation of agricultural fields in arid regions from Worldview-2 datasets based on image textural properties

Barren lands are being transformed into agricultural fields with the growing demand for agriculture-based products. Hence, monitoring these regions for better planning and management is crucial. Surveying with high-resolution RS (remote sensing) satellites like Worldview-2 provides a faster and cheaper solution than conventional surveys. In the study, the arid region comprising cropland and barrenlands are efficiently and autonomously delineated using its spectral and textural properties using state-of-the-art random forest (RF) ensemble classifiers. The textural information window size is optimized and at a GLCM (gray-level co-occurrence matrix) window size of 13, a stable trend in classification accuracy was observed. A further rise in window sizes did not improve the classification accuracy; beyond GLCM 19, a decline in accuracy was observed. Comparing GLCM-13 RF with the no-GLCM RF classifier, the GLCM-based classifiers performed better; thus, the textural information assisted in removing isolated crop-classified outputs that are falsely predicted pixel groups. Still, it also obscured information about barren lands present within croplands. Delineation accuracy was 93.8% for the no-GLCM RF classifier, whereas, for the GLCM-13 RF classifier, an accuracy of 97.3% was observed. Thus, overall, a 3.5% improvement in accuracy was observed while using the GLCM RF classifier with window size 13. The textural information with proper calibration over high-spatial resolution datasets improves crop delineation in the present study. Henceforth, a more accurate cropland identification will provide a better estimate of the actual cropland area in such an arid region, which will assist in formulating a better resource management policy.

Skeleton-based noise removal algorithm for binary concrete crack image segmentation

Image processing methods for automated concrete crack detection are often challenged by binary noise. Noise removal methods decrease the false positive pixels of crack detection results, often at the cost of a reduction in true positives. This paper proposes a novel method for binary noise removal and segmentation of noisy concrete crack images. The method applies an area threshold before reducing the pixel groups in the image to a skeleton. Each skeleton is connected to its nearest neighbour before the remaining short skeletons in the image are removed using a length threshold. A morphological reconstruction follows to remove all elements in the original noisy image that do not intersect with the skeleton. Finally, pixel groups in close proximity to the endpoints of the pixel groups in the resulting image are reinstated. Testing was conducted on a dataset of noisy binary crack images; the proposed method (Skele-Marker) obtained recall, precision, intersection over union, and F1 score results of 77%, 91%, 72%, and 84%, respectively. Skele-Marker was compared to other methods found in literature and was found to outperform other methods in terms of precision, intersection over union and F1 score. The proposed method is used to make crack detection results more reliable, supporting the ever-growing demand for automated inspections of concrete structures.

Characterization of pyrite weathering products by Raman hyperspectral imaging and chemometrics techniques

In this study, we propose a new approach based on Raman hyperspectral imaging and chemometrics for the analysis of chemically heterogeneous surfaces of weathered minerals. A pyrite sample showing a heterogeneous surface with different alteration products, is used to test the validity of the technique. Principal Component Analysis (PCA) was initially used to evaluate data structure and identify Raman phases and vibrational modes related to minor and major weathering features. Afterwards, Multivariate Curve Resolution-alternating least squares (MCR-ALS) was carried out to identify the specific chemical components of the major weathering phases. Finally, for the analysis of the minor weathering phases, K-means clustering was carried out to identify groups of similar pixels of low percentage of variance in the image. Three major components are found in the analyzed sample, corresponding to pyrite and hematite vibrational modes, as well as features of an amorphous alteration product patina. Four additional spectral signatures revealing the presence of sulfates were found after the analysis of minor components. The proposed method enables a semi-quantitative threshold-based characterization of chemical features and provides a visual representation of the phase distribution on the surface of the sample.

Preseason crop type prediction using crop sequence boundaries

Preseason crop-type prediction has emerged as a valuable tool for agricultural use. A reliable algorithm for early crop-type prediction has many applications, including crop mapping, planted acreage prediction, crop yield prediction, disaster response, area sample design, crop survey imputation, and more. The primary source of data for preseason crop prediction in the United States is the United States Department of Agriculture National Agricultural Statistics Service’s Cropland Data Layer, which is an annual crop specific land cover data set produced using satellite imagery and administrative data. Historical crop rotations taken from the Cropland Data Layer can be used by machine learning models to predict the future crop type in any given land area. The dataset obtained from the Cropland Data Layer is large, containing hundreds of millions of pixels per state. Current approaches for predictive modeling have utilized sampling, resulting in more scalable machine learning. In this paper, the authors propose an alternative method that uses all available Cropland Data Layer data in a rapid and memory-efficient manner. The proposed method relies on a novel technique for identifying groups of pixels with homogenous cropping history. These pixel groups are summarized as polygons representing field boundaries, referred to as crop sequence boundaries. Use of these new polygons for modeling significantly reduces the computational burden of incorporating all the Cropland Data Layer data and eliminates any increased uncertainty brought on by sampling. This novel polygon-based approach competes well with existing methods in scalability and accuracy, achieving the highest overall accuracy in 23 out of 24 tests performed.

Development of Automatic Inspection Systems for WRS2020 Plant Disaster Prevention Challenge Using Image Processing

In this article, an approach used for the inspection tasks in the WRS2020 Plant Disaster Prevention Challenge is explained. The tasks were categorized into three categories: reading pressure gauges, inspecting rust on a tank, and inspecting cracks in a tank. For reading pressure gauges, the “you only look once” algorithm was used to focus on a specific pressure gauge and check the pressure gauge range strings on the gauge using optical character recognition algorithm. Finally, a previously learned classifier was used to read the values shown in the gauge. For rust inspection, image processes were used to focus on a target plate that may be rusted for rust detection. In particular, it was necessary to report the rust area and distribution type. Thus, the pixel ratio and grouping of rust were used to count the rust. The approach for crack inspection was similar to that for rust. The target plate was focused on first, and then the length of the crack was measured using image processing. Its width was not measured but was calculated using the crack area and length. For each system developed to approach each task, the results of the preliminary experiment and those of WRS2020 are shown. Finally, the approaches are summarized, and planned future work is discussed.

Novel image cryptosystem based on new 2D hyperchaotic map and dynamical chaotic S-box

Chaotic systems are widely used in image encryption due to their sensitivity to initial values, ergodicity, and other properties; many image encryption algorithms based on chaotic systems have been studied in the past few years. To obtain a more secure encryption algorithm, this work firstly proposes a new two-dimensional discrete hyperchaotic map, which has a wider continuous chaotic interval, larger Lyapunov exponents and passed all NIST and part of TestU01 tests. Then, we apply the proposed map to generate S-boxes and combine them in pairs; finally, twelve S-boxes are obtained, and the elements of the plaintext image are grouped, each group of pixels is summed, and modular operations are used to specify specific S-boxes. Next, each set of elements is bitwise XOR with the corresponding S-box. Finally, the cipher image is obtained by scrambling using chaotic signal. Experiments show that compared with some other encryption algorithms, the proposed S-box-based encryption method has higher security, and it resists to common attacks.

Edge Detection Techniques Using Character Segmentation for OCR of Devanagari Text

An image processing technique called edge detection is used to find and identify the edges, curves, and objects in digital images. Edge detection includes various mathematical calculations that focus on locating the edges and curves where a digital image's brightness abruptly shifts or has discontinuities. In the fields of feature detection and feature extraction in image processing and computer vision, edge detection is a fundamental tool. The application of an edge detector results in the development of a set of connected curves that represent surface marks, character boundaries, and discontinuities in surface orientation. Character boundaries are described via edges. The majority of a feature is described by its edges. A group of pixels called an edge is used to describe an area where abrupt variations in intensity occur. Because it removes items of interest for actions like description, segmentation is a vital stage in an image recognition system. This paper focuses on edge detection techniques that can be implemented on characters for effective character recognition. Devanagari text includes curvy edges which need to be detected for proper recognition hence different edge detector operators are implemented to find the most effective amongst them to acquire clear edges of the text. There are several methods for segmenting images, including segmentation based on artificial neural networks, clustering, edges, regions, and thresholds. In this paper, edge-based segmentation is used. The Canny, Laplacian, Sobel, Prewitt, and Robert operators are used to detect edges, and the generated image is compared to the original binary image. The pre-processing phase includes the binarization step.

Two-Stage Cascaded CNN Model for 3D Mitochondria EM Segmentation

Mitochondria are the organelles that generate energy for the cells. Many studies have suggested that mitochondrial dysfunction or impairment may be related to cancer and other neurodegenerative disorders such as Alzheimer’s and Parkinson’s diseases. Therefore, morphologically detailed alterations in mitochondria and 3D reconstruction of mitochondria are highly demanded research problems in the performance of clinical diagnosis. Nevertheless, manual mitochondria segmentation over 3D electron microscopy volumes is not a trivial task. This study proposes a two-stage cascaded CNN architecture to achieve automated 3D mitochondria segmentation, combining the merits of top-down and bottom-up approaches. For top-down approaches, the segmentation is conducted on objects’ localization so that the delineations of objects’ contours can be more precise. However, the combinations of 2D segmentation from the top-down approaches are inadequate to perform proper 3D segmentation without the information on connectivity among frames. On the other hand, the bottom-up approach finds coherent groups of pixels and takes the information of 3D connectivity into account in segmentation to avoid the drawbacks of the 2D top-down approach. However, many small areas that share similar pixel properties with mitochondria become false positives due to insufficient information on objects’ localization. In the proposed method, the detection of mitochondria is carried out with multi-slice fusion in the first stage, forming the segmentation cues. Subsequently, the second stage is to perform 3D CNN segmentation that learns the pixel properties and the information of 3D connectivity under the supervision of cues from the detection stage. Experimental results show that the proposed structure alleviates the problems in both the top-down and bottom-up approaches, which significantly accomplishes better performance in segmentation and expedites clinical analysis.

End-to-End Implementation of a Convolutional Neural Network on a 3D-Integrated Image Sensor with Macropixel Array.

Three-dimensional-integrated focal-plane array image processor chips offer new opportunities to implement highly parallelised computer vision algorithms directly inside sensors. Neural networks in particular can perform highly complex machine vision tasks, and therefore their efficient implementation in such imagers are of significant interest. However, studies with existing pixel-processor array chips have focused on the implementation of a subset of neural network components-notably convolutional kernels-on pixel processor arrays. In this work, we implement a continuous end-to-end pipeline for a convolutional neural network from the digitisation of incoming photons to the output prediction vector on a macropixel processor array chip (where a single processor acts on group of pixels). Our implementation performs inference at a rate between 265 and 309 frames per second, directly inside of the sensor, by exploiting the different levels of parallelism available.

Boolean-based Two-in-One Secret Image Sharing by Adaptive Pixel Grouping

The two-in-one secret image sharing (TiOSIS) technique is a hybrid scheme that protects a secret image by combining visual cryptography (VCS) and polynomial-based secret image sharing (PSIS). There are two decoding methods available in TiOSIS: stacking-to-see decryption and lossless image recovery. However, the majority of current TiOSIS methods use Lagrange interpolation to precisely reconstruct the secret, which would result in intense computations. In this article, an efficient TiOSIS scheme using Boolean XOR operation for lossless image recovery is proposed. The proposed scheme consists of three building blocks: shared data generation, shadow construction, and image decryption. In shared data generation, the grayscale secret image is processed by a Boolean-based SIS to derive the shared bits. In shadow construction, an adaptive pixel grouping (APG) strategy is utilized to determine a grouping pattern. The halftone image adjustment algorithm is adopted to generate a suitable halftone image. With the grouping pattern and halftone image, we construct the shadows via the group-pixel embedding and sharing approach. In image decryption, we can reveal the secret image by stacking-to-see decoding or Boolean-based lossless image recovery. Extensive experiments and comparisons are illustrated to show the effectiveness and benefits of the proposed scheme.

Adaptive Noise Detector and Partition Filter for Image Restoration

The random-value impulse noise (RVIN) detection approach in image denoising, which is dependent on manually defined detection thresholds or local window information, does not have strong generalization performance and cannot successfully cope with damaged pictures with high noise levels. The fusion of the K-means clustering approach in the noise detection stage is reviewed in this research, and the internal relationship between the flat region and the detail area of the damaged picture is thoroughly explored to suggest an unique two-stage method for gray image denoising. Based on the concept of pixel clustering and grouping, all pixels in the damaged picture are separated into various groups based on gray distance similarity features, and the best detection threshold of each group is solved to identify the noise. In the noise reduction step, a partition decision filter based on the gray value characteristics of pixels in the flat and detail areas is given. For the noise pixels in flat and detail areas, local consensus index (LCI) weighted filter and edge direction filter are designed respectively to recover the pixels damaged by the RVIN. The experimental results show that the accuracy of the proposed noise detection method is more than 90%, and is superior to most mainstream methods. At the same time, the proposed filtering method not only has good noise reduction and generalization performance for natural images and medical images with medium and high noise but also is superior to other advanced filtering technologies in visual effect and objective quality evaluation.

Wavelet-Guided Promotion-Suppression Transformer for Surface-Defect Detection.

Surface-defect detection aims to accurately locate and classify defect areas in images via pixel-level annotations. Different from the objects in traditional image segmentation, defect areas comprise a small group of pixels with random shapes, characterized by uncommon textures and edges that are inconsistent with the normal surface patterns of industrial products. This task-specific knowledge is hardly considered in the current methods. Therefore, we propose a two-stage "promotion-suppression" transformer (PST) framework, which explicitly adopts the wavelet features to guide the network to focus on the detailed features in the images. Specifically, in the promotion stage, we propose the Haar augmentation module to improve the backbone's sensitivity to high-frequency details. However, the background noise is inevitably amplified as well because it also constitutes high-frequency information. Therefore, a quadratic feature-fusion module (QFFM) is proposed in the suppression stage, which exploits the two properties of noise: independence and attenuation. The QFFM analyzes the similarities and differences between noise and defect features to achieve noise suppression. Compared with the traditional linear-fusion approach, the QFFM is more sensitive to high-frequency details; thus, it can afford highly discriminative features. Extensive experiments are conducted on three datasets, namely DAGM, MT, and CRACK500, which demonstrate the superiority of the proposed PST framework.

F-TPE: Flexible Thumbnail-Preserving Encryption Based on Multi-Pixel Sum-Preserving Encryption

People are accustomed to utilizing mobile phones to capture images and uploading them to the cloud due to various incomparable advantages such as saving the storage space on the device. In this context, privacy concerns are raised since images may contain some sensitive information. Encrypting images by traditional schemes can alleviate privacy leak, but the usability is often deprived since it is tough to browse them on the cloud. Recently, Tajik <i>et al.</i> proposed an ideal <i>thumbnail-preserving encryption</i> (TPE) scheme to achieve the balance of usability and privacy by leveraging a two-pixel substitution encryption method while the Markov chain is utilized to prove the security. However, the connectivity of Markov chain in this scheme is weak since the length of pixel groups in the encryption process is only two, leading to a long mixing time of achieving the stationary distribution of Markov chain. To this end, we firstly propose a method of <i>multi-pixel sum-preserving encryption</i> (MP-SPE) that realizes the encipherment of vectors of arbitrary length. Then, with the help of MP-SPE, a novel flexible ideal TPE scheme (F-TPE) is designed and the connection of Markov chain is improved. The experiments have demonstrated that the proposed scheme can effectively attain the balance between usability and privacy. In addition, F-TPE takes much less time in encrypting compared with the existing work.

An Improved Steganographic Scheme Using the Contour Principle to Ensure the Privacy of Medical Data on Digital Images

With the improvement of current online communication schemes, it is now possible to successfully distribute and transport secured digital Content via the communication channel at a faster transmission rate. Traditional steganography and cryptography concepts are used to achieve the goal of concealing secret Content on a media and encrypting it before transmission. Both of the techniques mentioned above aid in the confidentiality of feature content. The proposed approach concerns secret content embodiment in selected pixels on digital image layers such as Red, Green, and Blue. The private Content originated from a medical client and was forwarded to a medical practitioner on the server end through the internet. The K-Means clustering principle uses the contouring approach to frame the pixel clusters on the image layers. The content embodiment procedure is performed on the selected pixel groups of all layers of the image using the Least Significant Bit (LSB) substitution technique to build the secret Content embedded image known as the stego image, which is subsequently transmitted across the internet medium to the server end. The experimental results are computed using the inputs from “Open-Access Medical Image Repositories (aylward.org)” and demonstrate the scheme’s impudence as the Content concealing procedure progresses.

Differentiable Multi-Granularity Human Parsing.

In this work, we study the challenging problem of instance-aware human body part parsing. We introduce a new bottom-up regime which achieves the task through learning category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner. The output is a compact, efficient and powerful framework that exploits structural information over different human granularities and eases the difficulty of person partitioning. Specifically, a dense-to-sparse projection field, which allows explicitly associating dense human semantics with sparse keypoints, is learnt and progressively improved over the network feature pyramid for robustness. Then, the difficult pixel grouping problem is cast as an easier, multi-person joint assembling task. By formulating joint association as maximum-weight bipartite matching, we develop two novel algorithms based on projected gradient descent and unbalanced optimal transport, respectively, to solve the matching problem differentiablly. These algorithms make our method end-to-end trainable and allow back-propagating the grouping error to directly supervise multi-granularity human representation learning. This is significantly distinguished from current bottom-up human parsers or pose estimators which require sophisticated post-processing or heuristic greedy algorithms. Extensive experiments on three instance-aware human parsing datasets (i.e., MHP-v2, DensePose-COCO, PASCAL-Person-Part) demonstrate that our approach outperforms most existing human parsers with much more efficient inference. Our code is available at https://github.com/tfzhou/MG-HumanParsing.

Enhancing UAV images to improve the observation of submerged whales using a water column correction method

Enhancing <scp>UAV</scp> images to improve the observation of submerged whales using a water column correction method

Local-Moment-Driven Robust Reversible Data Hiding

In this paper, a local-moment-driven robust reversible data hiding (LM-RRDH) scheme is proposed, which can provide security to hidden messages against unintentional modifications. The proposed LM-RRDH decomposes an image into LSB and MSB planes and then embeds the secret information into the MSB image so that intrusion by unintentional modifications can be avoided. In addition, the proposed scheme utilizes the prevalent correlation among the pixels on the MSB plane for optimal embedding. In the proposed scheme, a cover image is partitioned into sub-blocks at first, and pixel groups in the sub-block are formed according to local moment and moment-of-moment so that similar-intensity pixels can be grouped into the same group. Next, the secret data is embedded into the pixels of each group by selecting a pairwise embedding strategy adaptively which is based on the number of pixels in each group. As a result, the proposed LM-RRDH can limit the distortion while providing a decent embedding capacity. Further, a protection against non-malicious attacks such as Joint Photographic Experts Group (JPEG) compression is also provided. The experimental results show that the proposed scheme provides a superior quality to the previous works while providing a comparable embedding capacity.

Optimized Sample Adaptive Offset Filter in HEVC

The high-efficiency video coding (HEVC) standard incorporates sample adaptive offset (SAO) as an in-loop filtering technique to reduce distortion. The fundamental tenet of SAO is to classify every reconstructed pixel and then simply add the offset to each group of pixels. The previous existing technique employs 32 bands in band offset (BO) calculation to determine the proper offset value to minimize the distortion in the SAO estimate algorithm for statistics collection and parametric determination procedure. The suggested method will reduce the necessary band count by 16 bands. It does not alter how edge offset is determined in any way. It offers enough and efficient room for creating superior band group predictions and selections, as well as an ideal decrease in the implemented area. Thus, reducing the number of bands to 16 also decreases the amount of storage needed to hold each pixel’s information in binary files by up to 0.65%.