Pixel Gradient Research Articles

DCFNet: Infrared and Visible Image Fusion Network Based on Discrete Wavelet Transform and Convolutional Neural Network.

Aiming to address the issues of missing detailed information, the blurring of significant target information, and poor visual effects in current image fusion algorithms, this paper proposes an infrared and visible-light image fusion algorithm based on discrete wavelet transform and convolutional neural networks. Our backbone network is an autoencoder. A DWT layer is embedded in the encoder to optimize frequency-domain feature extraction and prevent information loss, and a bottleneck residual block and a coordinate attention mechanism are introduced to enhance the ability to capture and characterize the low- and high-frequency feature information; an IDWT layer is embedded in the decoder to achieve the feature reconstruction of the fused frequencies; the fusion strategy adopts the l1-norm fusion strategy to integrate the encoder's output frequency mapping features; a weighted loss containing pixel loss, gradient loss, and structural loss is constructed for optimizing network training. DWT decomposes the image into sub-bands at different scales, including low-frequency sub-bands and high-frequency sub-bands. The low-frequency sub-bands contain the structural information of the image, which corresponds to the important target information, while the high-frequency sub-bands contain the detail information, such as edge and texture information. Through IDWT, the low-frequency sub-bands that contain important target information are synthesized with the high-frequency sub-bands that enhance the details, ensuring that the important target information and texture details are clearly visible in the reconstructed image. The whole process is able to reconstruct the information of different frequency sub-bands back into the image non-destructively, so that the fused image appears natural and harmonious visually. Experimental results on public datasets show that the fusion algorithm performs well according to both subjective and objective evaluation criteria and that the fused image is clearer and contains more scene information, which verifies the effectiveness of the algorithm, and the results of the generalization experiments also show that our network has good generalization ability.

Crater Triangle Matching Algorithm Based on Fused Geometric and Regional Features

Craters are regarded as significant navigation landmarks during the descent and landing process in small body exploration missions for their universality. Recognizing and matching craters is a crucial prerequisite for visual and LIDAR-based navigation tasks. Compared to traditional algorithms, deep learning-based crater detection algorithms can achieve a higher recognition rate. However, matching crater detection results under various image transformations still poses challenges. To address the problem, a composite feature-matching algorithm that combines geometric descriptors and region descriptors (extracting normalized region pixel gradient features as feature vectors) is proposed. First, the geometric configuration map is constructed based on the crater detection results. Then, geometric descriptors and region descriptors are established within each feature primitive of the map. Subsequently, taking the salience of geometric features into consideration, composite feature descriptors with scale, rotation, and illumination invariance are generated through fusion geometric and region descriptors. Finally, descriptor matching is accomplished by computing the relative distances between descriptors and adhering to the nearest neighbor principle. Experimental results show that the composite feature descriptor proposed in this paper has better matching performance than only using shape descriptors or region descriptors, and can achieve a more than 90% correct matching rate, which can provide technical support for the small body visual navigation task.

Anti-Adversarially Manipulated Attributions for Weakly Supervised Semantic Segmentation and Object Localization.

Obtaining accurate pixel-level localization from class labels is a crucial process in weakly supervised semantic segmentation and object localization. Attribution maps from a trained classifier are widely used to provide pixel-level localization, but their focus tends to be restricted to a small discriminative region of the target object. An AdvCAM is an attribution map of an image that is manipulated to increase the classification score produced by a classifier before the final softmax or sigmoid layer. This manipulation is realized in an anti-adversarial manner, so that the original image is perturbed along pixel gradients in directions opposite to those used in an adversarial attack. This process enhances non-discriminative yet class-relevant features, which make an insufficient contribution to previous attribution maps, so that the resulting AdvCAM identifies more regions of the target object. In addition, we introduce a new regularization procedure that inhibits the incorrect attribution of regions unrelated to the target object and the excessive concentration of attributions on a small region of the target object. Our method achieves a new state-of-the-art performance in weakly and semi-supervised semantic segmentation, on both the PASCAL VOC 2012 and MS COCO 2014 datasets. In weakly supervised object localization, it achieves a new state-of-the-art performance on the CUB-200-2011 and ImageNet-1K datasets.

Feature Extraction Based on Local Histogram with Unequal Bins and a Recurrent Neural Network for the Diagnosis of Kidney Diseases from CT Images.

Kidney disease remains one of the most common ailments worldwide, with cancer being one of its most common forms. Early diagnosis can significantly increase the good prognosis for the patient. The development of an artificial intelligence-based system to assist in kidney cancer diagnosis is crucial because kidney illness is a global health concern, and there are limited nephrologists qualified to evaluate kidney cancer. Diagnosing and categorising different forms of renal failure presents the biggest treatment hurdle for kidney cancer. Thus, this article presents a novel method for detecting and classifying kidney cancer subgroups in Computed Tomography (CT) images based on an asymmetric local statistical pixel distribution. In the first step, the input image is non-overlapping windowed, and a statistical distribution of its pixels in each cancer type is built. Then, the method builds the asymmetric statistical distribution of the image's gradient pixels. Finally, the cancer type is identified by applying the two built statistical distributions to a Deep Neural Network (DNN). The proposed method was evaluated using a dataset collected and authorised by the Dhaka Central International Medical Hospital in Bangladesh, which includes 12,446 CT images of the whole abdomen and urogram, acquired with and without contrast. Based on the results, it is possible to confirm that the proposed method outperformed state-of-the-art methods in terms of the usual correctness criteria. The accuracy of the proposed method for all kidney cancer subtypes presented in the dataset was 99.89%, which is promising.

Goblet cells segmentation from confocal laser endomicroscopy with an improved U-Net

Gastric intestinal metaplasia (GIM) is regarded as a remarkable precursor for the development of intestinal-type stomach cancer. Goblet cell (GC) segmentation is the crucial step for assessing the degree of GIM by confocal laser endomicroscopy (CLE). However, GC segmentation by hand is difficult, unreliable, and time-consuming. Meanwhile, due to the high resolution and noise interference of CLE images, existing segmentation approaches perform poorly on this task. To tackle those issues, we collected 343 confocal laser endomicroscopy images of 62 patients from a Grade-A tertiary hospital. Each CLE image is manually annotated and then verified three times by skilled medical specialists. Then, U-Net is improved by incorporating the pixel gradient attention mechanism, which focuses on color gradient information around GC and captures color gradient features to direct feature maps in the skip connection layer. At last, the model output vector is used to calculate the possibility map and generate the final segmentation area. Compared with mainstream models, our proposed GC segmentation method from CLE with an improved U-Net (GCSCLE) performs the better segmentation result when tested on our CLE dataset and achieved an IOU of 87.95% and a DICE of 86.64%. Our result shows, the performance of the GCSCLE can be compared with the manual CLE image processing in clinical settings, and it can improve segmentation accuracy and save time and costs.

Discriminant Analysis Using Gabor Filter Sets for Lobular Endocervical Glandular Hyperplasia: Numerical Interpretation of Nuclear Atypia by Gabor Filter Features

Introduction: Lobular endocervical glandular hyperplasia (LEGH) is a benign lesion; however, it is considered to be the origin of gastric-type adenocarcinoma in the uterine cervix, and early diagnosis is important. At Shinshu University Hospital, screening of LEGH cells is based on the difference in color tone of cytoplasmic mucin on Papanicolaou staining and detection of gastric mucin using HIK1083-labeled latex agglutination assay. However, it is sometimes difficult to distinguish LEGH cells with subtle nuclear atypia from endocervical (EC) cells. Methods: We calculated the Gabor filter features (mean signal value, standard deviation, skewness, kurtosis) from the nuclei of cytological specimens in EC cells (37 cases) and LEGH cells (33 cases) using microscopic images, and we performed statistical analysis and discriminant analysis by linear support vector machine (LSVM) using these features. A Gabor filter is a linear filter defined as a mathematical representation of the mammalian visual system. Gabor filters with three wavelengths and eight angles were used for analysis. Results: Gabor filter features in EC cells were higher than in LEGH cells, demonstrating that the gradient of LEGH cell nuclei was milder than that of EC cell nuclei. The accuracy calculated using all Gabor filters was 91.0% and the accuracy of four Gabor filters (λ = 2/3π and θ = 0°, 45°, 90°, 135°) was 88.9%. High accuracy with low computation costs was achieved by reducing the number of features used for LSVM. Conclusion: The application of a Gabor filter with convolutional processing resulted in the edges of LEGH cells being slightly rough and thick, whereas those of EC cells were fine and thin. Thus, it is thought that the frequency of abrupt gradients of pixels was higher in EC cells than in LEGH cells, and the gradient of chromatin distribution in LEGH cell nuclei was milder than that in EC cell nuclei. It was possible to evaluate nuclear findings of EC and LEGH cells objectively by quantifying morphological features of nuclei using Gabor filters. It was possible to differentiate EC cells from LEGH cells using LSVM using Gabor filter features.

Wave force measurement on a large-scale vertical cylinder using a visual technique

Visual techniques were used to identify the wave force on a large-scale vertical cylinder for the first time in this study by tracking the dynamic waterlines around the cylinder. Experimental validation of the proposed method with two types of waves, regular and irregular, was conducted in a wave flume. Four cameras were used to capture images synchronously, and the dynamic waterlines around the cylinder were identified using a grayscale gradient of pixels. Based on the potential theory, the wave force, including the linear and part of the second- and higher-order force components, was identified using recursive least squares (RLS) in real-time. The identified wave force agreed well with the wave force measured by the load cells, particularly the linear wave forces. The identified results could be significantly improved by considering the nonlinear wave forces. The proposed method has broad applications for laboratory measurements of wave loads on cylindrical structures.

TVN: Detect Deepfakes Images using Texture Variation Network

Face manipulation technology is rapidly developing, making it impossible for human eyes to recognize fake face photos. Convolutional Neural Network (CNN) discriminators, on the other hand, can fast achieve high accuracy in distinguishing fake/real face photos. In this paper, we look at how CNN models discern between fake and real faces. Face forgery detection relies heavily on Texture Variation Network (TVN) information, according to our findings. We propose a new model, TVN, for robust face fraud detection, based on Convolution and pyramid pooling (PP), as a result of the aforesaid discovery. To produce a stationary representation of composition difference information, Convolution combines pixel intensity and pixel gradient information. Simultaneously, multi-scale information fusion based on the PP can prevent the texture features from being destroyed. Our TVN beats previous techniques on numerous databases, including Faceforensics++, DeeperForensics-1.0, Celeb-DF, and DFDC. The TVN is more resistant to image distortion, such as JPEG compression and blur, which is critical in the wild.

Facial expression recognition of aerobics athletes based on CNN and HOG dual channel feature fusion

The problem of low feature extraction accuracy and low recognition accuracy in facial expression recognition of aerobics athletes is presented. A recognition method for fusion CNN and HOG dual channel features is proposed. The basic principle of the HOG is analysed, and the facial expression image of aerobics athletes is processed by grey level with the help of local binary mode. The pixel gradient intensity value in each small image is obtained, and all the intensity values are fused. Lagrange formula is used to transform high-dimensional features. Support vector machine is used to classify facial expression images, and feature points are used as CNN, to process feature points according to network input. Regularisation regression is used to realise facial expression recognition of aerobics athletes. The experimental results show that the accuracy of feature extraction is 97% and the recognition accuracy is always higher than 90%.

Machine vision-based transverse vibration measurement of diamond wire

Diamond wire sawing is one of the key technologies for wafering in the semiconductor industry. The transverse vibration of the wire during sawing will generate additional displacement that not only reduces the as-cut wafer quality but increases the critical adhesion gap between the wires, which is an important factor limiting the development of diamond wire slicing technology toward narrower kerf loss and thinner as-cut wafer. The vibration measurement of the diamond wire provides an indispensable test basis for its vibration control. Therefore, a machine vision-based transverse vibration measurement method of diamond wire is developed in this paper. The approximate edge position of the wire in the image midline is obtained to crop the region of interest in the image by using the maximum continuous same-directional pixel gradient algorithm and thus reduce the computing time. The wire edges in the image are accurately extracted by an adaptive image segmentation method proposed based on the image histogram. According to the surface topography characteristics of diamond wire, a feature extraction algorithm based on the multiple inward line fitting using the least square method is proposed, which can effectively detect the position of the inner diameter boundary of the diamond wire. Accordingly, the equation of the central axis of the wire in the image is obtained to realize the transverse vibration detection of the diamond wire. To conclude, a diamond wire vibration measurement experiment is conducted and the machine vision detection results are compared with the vibration displacement measured by the accelerometer at the same position of the wire end. The results show that the measurement error of the machine vision vibration inspection is generally small for the different types of diamond wires and the different frequencies of the wire transverse vibration, which means that the measurement method proposed in this paper has good accuracy, and applicability, and robustness.

Dust pollution evaluation based on grayscale average and fractal dimension of digital image

The automatic and effective identification of mine dust draws growing public concern for its universality and being highly hazardous to health and property. Based on improved methods for improved grayscale average (IGSA) and fractal dimension (FD) theory, a vision-based system that employs digital image processing was proposed to recognize dust particles. The proposed approach primarily includes the following procedures: image matrix generation; pixel gradient calculations; image processing for sharpening, gray transformation, and binarization; IGSA and FD calculations, and visualization accomplishments. Image example results show that FD appears to have an exponential relationship with IGSA as well as dust pollution. Then, a dust pollution-level evaluation method based on the pollution-factor Fp was established and contributed to the division of dust pollution regions. During the experiment, wind speed and hole distance were proved to demonstrate a positive and negative correlation with IGSA, respectively, while little impact was observed on the FD and Fp with the factors. A dust pollution monitoring system that is theoretically viable was developed to form an Industrial Internet of Things with a topological structure. The proposed approach provides a digital image-processing method to integrally and automatically characterize the dust morphology. This approach can achieve dust pollution-level and regional divisions.

Infrared and Visible Image Fusion Based on Visual Saliency Map and Image Contrast Enhancement.

The purpose of infrared and visible image fusion is to generate images with prominent targets and rich information which provides the basis for target detection and recognition. Among the existing image fusion methods, the traditional method is easy to produce artifacts, and the information of the visible target and texture details are not fully preserved, especially for the image fusion under dark scenes and smoke conditions. Therefore, an infrared and visible image fusion method is proposed based on visual saliency image and image contrast enhancement processing. Aiming at the problem that low image contrast brings difficulty to fusion, an improved gamma correction and local mean method is used to enhance the input image contrast. To suppress artifacts that are prone to occur in the process of image fusion, a differential rolling guidance filter (DRGF) method is adopted to decompose the input image into the basic layer and the detail layer. Compared with the traditional multi-scale decomposition method, this method can retain specific edge information and reduce the occurrence of artifacts. In order to solve the problem that the salient object of the fused image is not prominent and the texture detail information is not fully preserved, the salient map extraction method is used to extract the infrared image salient map to guide the fusion image target weight, and on the other hand, it is used to control the fusion weight of the basic layer to improve the shortcomings of the traditional ‘average’ fusion method to weaken the contrast information. In addition, a method based on pixel intensity and gradient is proposed to fuse the detail layer and retain the edge and detail information to the greatest extent. Experimental results show that the proposed method is superior to other fusion algorithms in both subjective and objective aspects.

Synchronous monitoring of axial vibration and rotation speed of rotating cylinder by linear array scanning

The axial vibration (AV) and rotation speed (RS) signals of a rotating cylinder contain rich condition information. In order to harness such quantitative information, an effective measurement method for synchronous monitoring of the AV and RS is highly desirable. However, traditional approaches are restricted either by limited test distance or multiple isolated sensors, which makes the approaches impractical to synchronously acquire the AV and RS signals through a single device. In this paper, a generalized visual measurement method for synchronous monitoring of the AV and RS via linear array scanning (LAS) is proposed. The basic idea is to paste a mark containing a printed black continuous bar and a black block (BCBB) on the surface of a rotating cylinder, and use a linear array camera to synchronously sample the bar and blocks. For the extraction of mark edges, we propose a single edge pixel extraction method through gray gradient of row pixels based on the observation of relatively stable gray level distribution. Line scanned signals disturbed by seam error or axial offset are automatically corrected through seam compensation and axial offset correction respectively. Experimental validation of the proposed method is presented for the cylinder under static hammering, variable speed condition and steady speed condition at a scanning distance of 500 mm. Bilinear interpolation is applied to the scanned linear array images before extracting the edge features and helps improve measurement accuracy. Analyses in the time domain and frequency domain verify the effectiveness of the LAS. The average error between the RS signal obtained by LAS and the reference value is 0.1420 rpm. The correlation coefficient between the AV time-domain signal of LAS and the reference signal is up to 0.900 and the resonance frequency error in the frequency spectrum is less than 1 Hz.

Local Label Point Correction for Edge Detection of Overlapping Cervical Cells.

Accurate labeling is essential for supervised deep learning methods. However, it is almost impossible to accurately and manually annotate thousands of images, which results in many labeling errors for most datasets. We proposes a local label point correction (LLPC) method to improve annotation quality for edge detection and image segmentation tasks. Our algorithm contains three steps: gradient-guided point correction, point interpolation, and local point smoothing. We correct the labels of object contours by moving the annotated points to the pixel gradient peaks. This can improve the edge localization accuracy, but it also causes unsmooth contours due to the interference of image noise. Therefore, we design a point smoothing method based on local linear fitting to smooth the corrected edge. To verify the effectiveness of our LLPC, we construct a largest overlapping cervical cell edge detection dataset (CCEDD) with higher precision label corrected by our label correction method. Our LLPC only needs to set three parameters, but yields 30–40% average precision improvement on multiple networks. The qualitative and quantitative experimental results show that our LLPC can improve the quality of manual labels and the accuracy of overlapping cell edge detection. We hope that our study will give a strong boost to the development of the label correction for edge detection and image segmentation. We will release the dataset and code at: https://github.com/nachifur/LLPC.

Dressi: A Hardware‐Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization

AbstractDifferentiable rendering (DR) enables various computer graphics and computer vision applications through gradient‐based optimization with derivatives of the rendering equation. Most rasterization‐based approaches are built on general‐purpose automatic differentiation (AD) libraries and DR‐specific modules handcrafted using CUDA. Such a system design mixes DR algorithm implementation and algorithm building blocks, resulting in hardware dependency and limited performance. In this paper, we present a practical hardware‐agnostic differentiable renderer called Dressi, which is based on a new full AD design. The DR algorithms of Dressi are fully written in our Vulkan‐based AD for DR, Dressi‐AD, which supports all primitive operations for DR. Dressi‐AD and our inverse UV technique inside it bring hardware independence and acceleration by graphics hardware. Stage packing, our runtime optimization technique, can adapt hardware constraints and efficiently execute complex computational graphs of DR with reactive cache considering the render pass hierarchy of Vulkan. HardSoftRas, our novel rendering process, is designed for inverse rendering with a graphics pipeline. Under the limited functionalities of the graphics pipeline, HardSoftRas can propagate the gradients of pixels from the screen space to far‐range triangle attributes. Our experiments and applications demonstrate that Dressi establishes hardware independence, high‐quality and robust optimization with fast speed, and photorealistic rendering.

Research on Massive Image Retrieval Method of Mobile Terminal Based on Weighted Aggregation Depth Feature

Image self-coupling and feature interference lead to poor retrieval performance in massive image retrieval of mobile terminals. This paper proposes a massive image retrieval method of mobile terminals based on weighted aggregation depth features. The pixel big data detection model of massive images of mobile terminals is constructed, the collected pixel information of massive images of mobile terminals is restructured, the edge contour feature parameter set of massive images of mobile terminals is extracted, the feature fusion processing of massive images of mobile terminals is carried out in gradient pixel space by means of feature reconstruction and gray moment invariant feature analysis, the depth feature detection of massive images of mobile terminals is realized by using weighted aggregation method, the gradient value of pixels of massive images of each mobile terminal is calculated, and the optimized retrieval of massive images of mobile terminals is realized according to the fusion result of gradient weighted information. Simulation results show that this method has better feature clustering, stronger image detection and recognition, and anti-interference ability and improves the precision and recall of image retrieval.

Directional Difference Convolution and Its Application on Face Anti-Spoofing

In practical application, facial image recognition is vulnerable to be attacked by photos, videos, etc., while some currently used artificial feature extractors in machine learning, such as activity detection, texture descriptors, and distortion detection, are insufficient due to their weak detection ability in feature extraction from unknown attack. In order to deal with the aforementioned deficiency and improve the network security, this paper proposes directional difference convolution for the deep learning in gradient image information extraction, which analyzes pixel correlation within the convolution domain and calculates pixel gradients through difference calculation. Its combination with traditional convolution can be optimized by a parameter θ. Its stronger ability in gradient extraction improves the learning and predicting ability of the network, whose performance testing on CASIA-MFSD, Replay-Attack, and MSU-MFSD for face anti-spoofing task shows that our method outperforms the current related methods.

Tumor Classification Based on Regional Heterogeneity Using Pixel Level Feature Descriptors

Brain tumor is considered to be widely analyzed disease for effective diagnosis and treatment planning. Several approaches were framed to detect and diagnose tumor at early stage. In this work, texture analysis is carried out to analyze the nature of tumor and categorize it. Around 3064 images were analyzed during this study consisting of meningioma, glioma and pituitary tumors. Intensity and gradient pixel based texture analysis is carried out in this analysis. Results confirm that the tumors can be classified and categorized based on the intensity and gradient pixel information. A total of 2216 feature vector is extracted it is observed that the gradient based information aids for better classification of tumors. Localized binary patterns are found to provide detailed information about the subtle variation in the brain regions due to the presence of abnormality in brain tissues. It is further observed that the normalized feature vectors show better differentiation between tumor categories. The ROC and PRC curves exhibit the high classification ability using the extracted features to differentiate tumor grades.

Geometric Regularized Hopfield Neural Network for Medical Image Enhancement.

One of the major shortcomings of Hopfield neural network (HNN) is that the network may not always converge to a fixed point. HNN, predominantly, is limited to local optimization during training to achieve network stability. In this paper, the convergence problem is addressed using two approaches: (a) by sequencing the activation of a continuous modified HNN (MHNN) based on the geometric correlation of features within various image hyperplanes via pixel gradient vectors and (b) by regulating geometric pixel gradient vectors. These are achieved by regularizing proposed MHNNs under cohomology, which enables them to act as an unconventional filter for pixel spectral sequences. It shifts the focus to both local and global optimizations in order to strengthen feature correlations within each image subspace. As a result, it enhances edges, information content, contrast, and resolution. The proposed algorithm was tested on fifteen different medical images, where evaluations were made based on entropy, visual information fidelity (VIF), weighted peak signal-to-noise ratio (WPSNR), contrast, and homogeneity. Our results confirmed superiority as compared to four existing benchmark enhancement methods.

MTD-Net: Learning to Detect Deepfakes Images by Multi-Scale Texture Difference

With the rapid development of face manipulation technology, it is difficult for human eyes to distinguish fake face images. On the contrary, Convolutional Neural Network (CNN) discriminators can quickly reach high accuracy in identifying fake/real face images. In this study, we explore the behavior of CNN models in distinguish fake/real faces. We find multi-scale texture difference information plays an important role in face forgery detection. Motivated by the above observation, we propose a new Multi-scale Texture Difference model coined as MTD-Net for robust face forgery detection, which leverages central difference convolution (CDC) and atrous spatial pyramid pooling (ASPP). CDC combines the pixel intensity information and the pixel gradient information to give a stationary description of texture difference information. Simultaneously, based on the ASPP, multi-scale information fusion can keep the texture features from being destroyed. Experimental results on several databases, Faceforensics++, DeeperForensics-1.0, Celeb-DF and DFDC prove that our MTD-Net outperforms existing approaches. The MTD-Net is more robust to image distortion, e.g. , JPEG compression and blur, which is urgently needed in the wild world.