Global Image Information Research Articles

Towards No-Reference Image Quality Assessment Based on Multi-Scale Convolutional Neural Network

Image quality assessment has become increasingly important in image quality monitoring and reliability assuring of image processing systems. Most of the existing no-reference image quality assessment methods mainly exploit the global information of image while ignoring vital local information. Actually, the introduced distortion depends on a slight difference in details between the distorted image and the non-distorted reference image. In light of this, we propose a no-reference image quality assessment method based on a multi-scale convolutional neural network, which integrates both global information and local information of an image. We first adopt the image pyramid method to generate four scale images required for network input and then provide two network models by respectively using two fusion strategies to evaluate image quality. In order to better adapt to the quality assessment of the entire image, we use two different loss functions in the training and validation phases. The superiority of the proposed method is verified by several different experiments on the LIVE datasets and TID2008 datasets.

A Novel Hybrid Active Contour Model for Intracranial Tuberculosis MRI Segmentation Applications

This paper proposes an improved region-based active contour model for segmenting magnetic resonance imaging (MRI) images of brain tuberculosis by combining a global energy fitting term and a local energy fitting term. First, a global energy fitting term is utilized to extract global image information, which guides the evolving curve globally and approximates the image intensity inside and outside the contour. Second, a local energy fitting term is proposed to describe the intensity inhomogeneity based on the local intensity variance and the adaptive image difference. Third, an improved Fuzzy C-Means (FCM) clustering method is applied to pre-segment the MRI images to automatically track the approximate location of brain tuberculosis and provide the initial contour for the hybrid model segmentation. By combining the global and local weighting functions, a hybrid region-based model is defined. Experiments demonstrate that the proposed model provides initialization of the contours automatically and offers superior segmentation performance for brain tuberculosis MRI medical images with intensity inhomogeneity.

Hyperspectral image visualization with edge-preserving filtering and principal component analysis

In this paper, an edge-preserving filtering and principal component analysis (PCA)-based visualization method is proposed for hyperspectral images, in which both global and local image information of hyperspectral images (HSIs) are taken into account in the proposed visualization framework that consists of the following major steps. First, the band number of the original image is reduced with averaging-based image fusion (AIF). Then, the edge-preserving filtering is performed on the dimension reduced image so as to decompose it into two components, i.e., the base layers which contain the large-scale boundary information, and the detail layers which contain the mid- and small-scale edges and textures. Next, the base layers are fused with the principal component analysis method, and the detail layers are fused with the weighted sum method, in which the fusion weights are determined by the transform matrix of the base layers. Finally, the fused detail layers are enhanced with histogram equalization and combined with the fused base layers to visualize the hyperspectral image. Experimental results on four real hyperspectral data sets demonstrate that the proposed approach performs the best in improving image contrast and preserving the details. In addition, compared with seven state-of-the-art visualization methods, the quantitative metrics obtained by the proposed method on four data sets have been increased by 80.45%, 69.29%, 138.61%, and 23.21% on average in terms of separability of features (SF), standard deviation (SD), average gradient (AG), and entropy (EN).

Cross-scale fusion detection with global attribute for dense captioning

As a new task of image understanding, the dense caption model needs to locate and describe a salient region in the image simultaneously. It inevitably divides the dense caption model into two parts, one part for detecting the regions of interest and the other part for generating regional language caption. Previous methods are relatively simple to deal with these two parts, using the feature map on the last convolution layer of RPN network to predict object coordinates, and using LSTM for regional language modeling. However, the structure of RPN is insufficient to deal with a large number of objects in the complex dataset, and LSTM also fails to effectively utilize the global information of images in regional language training, which brings opportunity to improve the performance of dense caption. In this paper, we propose a novel Cross-scale Fusion with Global Attribute model (CSGA) that enables the two parts of the dense caption model to perform normal end-to-end training without mutual interference. Furthermore, our model uses a one-stage object detector with feature map fusion operation across multiple detection scales to improve the quality of object detection part, and combines image features with the global high-level attribute to improve regional language training. We design a variety of model architectures and conducted sufficient experiments. Empirical results on Visual Genome dataset show that our model achieves competitive results with mAP 8.33.

Aggregated Deep Fisher Feature for VHR Remote Sensing Scene Classification

With the development of very high resolution satellite image acquisition technology, remote sensing scene classification has become an important and challenging task. In this article, aiming at tackling this task, we propose a hybrid architecture, i.e., aggregated deep Fisher feature (ADFF), which can make full use of deep convolutional features’ rich semantic information and unsupervised encoding's high robustness. Unlike the previous methods, we first explore the optimal encoding layer in the pretraining CNN model, which naturally fuses the local and global image information in a novel way, making the ability of semantic acquisition further enhanced. ADFF can learn more suitable internal features from the remote sensing data, boosting the final performance. We evaluate our algorithm based on several public datasets, and the results show that our approach achieves superior performance compared with the state-of-the-art methods. The proposed ADFF obtains average classification accuracy of 98.81%, 95.21%, 86.01%, and 88.79%, respectively, on the UC Merced Land-Use, RSSCN7, NWPU-RESISC45 (10% for training), and NWPU-RESISC45 (20% for training) datasets.

Interactive Hierarchical Object Proposals

Object proposal algorithms have been demonstrated to be very successful in accelerating object detection process. High object localization quality and detection recall can be obtained using thousands of proposals. However, the performance with a small number of proposals is still unsatisfactory. This paper demonstrates that the performance of a few proposals can be significantly improved with the minimal human interaction—a single touch point. To this end, we first generate hierarchical superpixels using an efficient tree-organized structure as our initial object proposals, and then select only a few proposals from them by learning an effective Convolutional neural network for objectness ranking. We explore and design an architecture to integrate human interaction with the global information of the whole image for objectness scoring, which is able to significantly improve the performance with a minimum number of object proposals. Extensive experiments show the proposed method outperforms all the state-of-the-art methods for locating the meaningful object with the touch point constraint. Furthermore, the proposed method is extended for video. By combining with the novel interactive motion segmentation cue for generating hierarchical superpixels, the performance on a single proposal is satisfactory and can be used in the interactive vision systems, such as selecting the input of a real-time tracking system.

The ideal effect of Gabor filters and Uniform Local Binary Pattern combinations on deformed scanned paper images

Existing scanners produce paper images with different types of deformations such as noise, rotation and shear. These deformations affect the accuracy of the fingerprinting the document images, which entails utilizing advanced feature extraction operators. Existing feature extractor such as the Uniform Local Binary Patterns (ULBP) has been found to be limited in dealing with the global view of the texture and neglecting useful information about the images. This article presents an Automated Paper Fingerprinting (APF) method that deploys a combination approach for Gabor Filters (GF) and Uniform Local Binary Patterns (ULBP) called the GFULBP operator to cater for both local and global image information during the feature extraction process for higher texture classification accuracy. The APF method is evaluated by a standard dataset of 306 blank paper images derived from pre-existing scanner image dataset from Universiti Kebangsaan Malaysia (UKM) with properties ranges from 50 DPI, 100 DPI, and 150 DPI respectively. The images are captured by a flatbed scanner with 50 DPI, 100 DPI, and 150 DPI resolutions. Each image is represented by four patches that are segmented from specific locations of the image. The test results of the APF show that GFULBP is able to outperform the ULBP alone by 30.68% when the GF has a 5 scale and π/2 orientation degree. This work finds that the integration of Gabor filters and ULBP significantly enhances the feature extraction quality and fingerprinting accuracy.

Active contour model for satellitale image combining local and global region based approach

<p>Global and local image information is crucial for accurate segmentation of images with intensity inhomogeneity valuable minute details and multiple objects with various intensities. We propose a region-based active contour model which is able to utilize together local and global image information. The major contribution of this paper is to expand the LIF model which is includes only local image infofmation to a local and global model. The introduction of a new local and global signed pressure force function enables the extraction of accurate local and global image information and extracts multiple objects with several intensities. Several tests performed on some synthetic and real images indicate that our model is effective as well as less sensitivity to the initial contour location and less time compared with the related works. </p><p><em> </em></p>

A Hybride Active Contour Model Driven by Global and Local Image Information

In this paper, a novel active contour model based on hybrid image fitting energy which utilizes both global and local image information is proposed. Two fitting images are constructed to approximate the original image and the square of the original image. Both global and local image information are incorporated into these two fitting images. Based on these two fitting images, a hybrid image fitting energy, which is then minimized in a variational level set framework to guide the evolving contours to the desired boundaries. The proposed approach is validated by experiments on both synthetic and real images. The experiments demonstrate that the proposed model is more efficient and robust for segmenting different kinds of images compared with several typical active contour models.

Active Contour Driven by Weighted Hybrid Signed Pressure Force for Image Segmentation

This study presents a novel active contour model (ACM) driven by weighted global and local region-based signed pressure force (SPF) to segment images in the presence of intensity inhomogeneity and noise. First, an adaptive weighted global region-based SPF (GRSPF) function as the driving centers is designed based on the global image information, which is based on the normalized global intensity to update the weights of the inner and outer regions of the curve during iterations. Second, by introducing the normalized absolute local intensity differences as the weighs of the inner and outer regions, an adaptive weighted local region-based SPF (LRSPF) function is similarly defined. Third, instead of setting a fixed force, a force propagation function is introduced to automatically balance the interior and exterior forces according to the image feature. Meanwhile, by combing the adaptive GWSPF and LWSPF functions, a weighted hybrid region-based SPF function is defined, which can improve the efficiency and accuracy of the proposed model. The experimental results on real images demonstrate that the proposed model is more robust than the popular region-based ACMs for segmenting images with intensity inhomogeneity and noise. The code is available at https://github.com/fangchj2002/WHRSPF .

Hybrid Active Contours Driven by Edge and Region Fitting Energies Based on p-Laplace Equation

By using local and global image information, a novel active contour method based on the p-Laplace equation for image segmentation is proposed in this paper. The force term in the evolution equation incorporates global, local and edge information of the image. The global and local terms drag the contour towards object boundaries precisely and take care of the contour movement when it is away from the object. Meanwhile, the integration of edge stopping function in this force term ensures the stoppage of contour at object boundaries to avoid boundary leakage problem. The variable exponent p-Laplace energy is used for smoothness of the level set to detect the exact object boundaries in the presence of complex topological changes and deep depression. Finally, the adaptive force and p-Laplace energy term are jointly integrated into a level set by using a simple finite difference scheme to build the final evolution equation for the method. The proposed method has strong capability to accurately segment the images having noise, intensity inhomogeneity, and complex object boundaries. Moreover, the proposed method overcomes the problem arise from contour initialization as the evolution of contour is independent of the initialization of level set function. Experimental results on synthetic, real and medical images along with two publicly available databases validate the robustness and effectiveness of the proposed method.

An Image Segmentation Method Using an Active Contour Model Based on Improved SPF and LIF

Inhomogeneous images cannot be segmented quickly or accurately using local or global image information. To solve this problem, an image segmentation method using a novel active contour model that is based on an improved signed pressure force (SPF) function and a local image fitting (LIF) model is proposed in this paper, which is based on local and global image information. First, a weight function of the global grayscale means of the inside and outside of a contour curve is presented by combining the internal gray mean value with the external gray mean value, based on which a new SPF function is defined. The SPF function can segment blurred images and weak gradient images. Then, the LIF model is introduced by using local image information to segment intensity-inhomogeneous images. Subsequently, a weight function is established based on the local and global image information, and then the weight function is used to adjust the weights between the local information term and the global information term. Thus, a novel active contour model is presented, and an improved SPF- and LIF-based image segmentation (SPFLIF-IS) algorithm is developed based on that model. Experimental results show that the proposed method not only exhibits high robustness to the initial contour and noise but also effectively segments multiobjective images and images with intensity inhomogeneity and can analyze real images well.

Optimised transformation model for contrast enhancement of satellite images using modified whale optimisation

ABSTRACT Image enhancement is possible only when sufficient information about the concerned is on hand. This would also directly reflect on the quality of the image. Enhancement of satellite image is deductively conceivable by processing a given image using soft computing technique. This paper introduces an endeavour to exhibit its versatility and adequacy for searching global optimal solutions for improving the contrast properties of a given image. The intensity of the image is enhanced by a parameterised transformation function, in which parameters are advanced by modified whale optimisation (MWO) as an objective function. The intensity transformation function utilises local and global information of the input image and the objective function considers the entropy and edge data to measure the image quality.

Detect Globally, Label Locally: Learning Accurate 6-DOF Object Pose Estimation by Joint Segmentation and Coordinate Regression

Coordinate regression has established itself as one of the most successful current trends in model-based 6 degree of freedom (6-DOF) object pose estimation from a single image. The underlying idea is to train a system that can regress the three-dimensional coordinates of an object, given an input RGB or RGB-D image and known object geometry, followed by a robust procedure such as RANSAC to optimize the object pose. These coordinate regression based approaches exhibit state-of-the-art performance by using pixel-level cues to model the probability distribution of object parts within the image. However, they fail to capture global information at the object level to learn accurate foreground/background segmentation. In this letter, we show that combining global features for object segmentation and local features for coordinate regression results in pixel-accurate object boundary detections and consequently a substantial reduction in outliers and an increase in overall performance. We propose a deep architecture with an instance-level object segmentation network that exploits global image information for object/background segmentation and a pixel-level classification network for coordinate regression based on local features. We evaluate our approach on the standard ground-truth 6-DOF pose estimation benchmarks and show that our joint approach to accurate object segmentation and coordinate regression results in the state-of-the-art performance on both RGB and RGB-D 6-DOF pose estimation.

A fast and reliable noise-resistant medical image segmentation and bias field correction model

In recent years, with the rapid development of modern medical image technology, the medical image processing technology is becoming more important. In particular, the accurate segmentation of medical images is significant for doctors to diagnose and analyze the etiology. However, the false contours appearing in medical images due to fuzzy image boundary, intensity inhomogeneity and random noise, may lead to the inaccurate segmentation results. In this paper, an improved active contour model based on global image information is proposed, which can accurately segment images disturbed by intensity inhomogeneities and serious noise. We give the two-phase energy functional and multi-phase energy functional of our model, and apply it to segment magnetic resonance (MR) images, ultrasound (US) images and synthetic images. Experimental results and comparisons with other models have shown that our model has the advantages of higher accuracy, higher efficiency and robustness in dealing with the intensity inhomogeneity and serious noise in image segmentation.

Adaptive active contours based on variable kernel with constant initialisation

In this paper, a novel method of active contours based on the formulation of partial differential equation (PDE) is proposed for image segmentation. The evolution equation incorporates a force term that pushes the contour towards object boundary, a regularisation term which takes into account the smoothness of the level set function and an edge term which helps to stop the contour at required boundaries. The proposed method integrates an image convolved by a variable kernel into an energy formulation, where the width of the kernel varies in each iteration. Therefore, it takes local region information when the width of the kernel is small while for the larger width of the kernel, the proposed method considers global region information across the regions. Due to the use of both local and global image information, the method easily detects objects in the complex background and also segments the objects where intensity changes within the object. Moreover, the proposed method totally eliminates the need of the contour initialisation by using constant initialisation scheme. Experimental results on real and medical images prove the robustness of the proposed method. Finally, the authors validate their method on PH2 database for skin lesion segmentation.

An optimization framework of video advertising: using deep learning algorithm based on global image information

Object detection and recognition technology plays an important role in video advertising. Based on the deep learning algorithm, this paper first builds a real-time object detection model (Yes-Net) based on image global information which combines the advantages of CNN and RNN algorithm together and has achieved significant effects compared with other object detection algorithms. We used this model to build an accurate video advertising framework and applied the framework in an IPTV platform of a telecommunication operator to collect comparative data. We first performed image detection and recognition in video playing, and then associated the recognition results with the ads database to conduct real-time classification and accurate ads delivery. It proves that compared with traditional advertisement delivery method, this new framework dramatically improves the click-through rate, and achieves better results.

A novel character segmentation method for serial number on banknotes with complex background

The serial number recognition has played a crucial role in financial market. And the character segmentation is the most important part for serial number recognition. In this paper, in order to deal with the noises and complex textures of the banknotes, we propose a novel method which is composed of a hybrid binarization algorithm (HybridB) and an adaptive character extraction algorithm (ACE). The HybridB algorithm utilizes both the global information of the whole image as well as the local information of the pixels which can eliminate the interferences of the noises and background. The ACE algorithm adjusts the boundaries of each character by the local contrast average value information and the distance between its gravity and center, and a decreasing step strategy is also employed to optimize the boundaries to get more accurate segmentation results. To validate the performance of the proposed method, we take experiments on the datasets of RMB, GBP and USD. The binarization results and character extraction results show that our proposed method outperforms the other state-of-the-art algorithms in most cases.

Learning Multi-Modality Features for Scene Classification of High-Resolution Remote Sensing Images

Scene classification of high-resolution remote sensing (HRRS) image is an important research topic and has been applied broadly in many fields. Deep learning method has shown its high potential to in this domain, owing to its powerful learning ability of characterizing complex patterns. However the deep learning methods omit some global and local information of the HRRS image. To this end, in this article we show efforts to adopt explicit global and local information to provide complementary information to deep models. Specifically, we use a patch based MS-CLBP method to acquire global and local representations, and then we consider a pretrained CNN model as a feature extractor and extract deep hierarchical features from full-connection layers. After fisher vector (FV) encoding, we obtain the holistic visual representation of the scene image. We view the scene classification as a reconstruction procedure and train several class-specific stack denoising autoencoders (SDAEs) of corresponding class, i.e., one SDAE per class, and classify the test image according to the reconstruction error. Experimental results show that our combination method outperforms the state-of-the-art deep learning classification methods without employing fine-tuning.

Multi-scale multi-class conditional generative adversarial network for handwritten character generation

Handwritten character generation is a popular research topic with various applications. Various methods have been proposed in the literatures which are based on methods such as pattern recognition, machine learning, deep learning or others. However, seldom method could generate realistic and natural handwritten characters with a built-in determination mechanism to enhance the quality of generated image and make the observers unable to tell whether they are written by a person. To address these problems, in this paper, we proposed a novel generative adversarial network, multi-scale multi-class generative adversarial network (MSMC-CGAN). It is a neural network based on conditional generative adversarial network (CGAN), and it is designed for realistic multi-scale character generation. MSMC-CGAN combines the global and partial image information as condition, and the condition can also help us to generate multi-class handwritten characters. Our model is designed with unique neural network structures, image features and training method. To validate the performance of our model, we utilized it in Chinese handwriting generation, and an evaluation method called mean opinion score (MOS) was used. The MOS results show that MSMC-CGAN achieved good performance.