Multi-scale Block Research Articles

Face morph detection for unknown morphing algorithms and image sources: a multi‐scale block local binary pattern fusion approach

The vulnerability of face recognition systems against so-called morphing attacks has been revealed in the past years. Recently, different kinds of morphing attack detection approaches have been proposed. However, the vast majority of published results has been obtained from rather constrained experimental setups. In particular, most investigations do not consider variations in morphing techniques, image sources, and image post-processing. Hence, reported performance rates can not be maintained in realistic scenarios, as the NIST FRVT MORPH performance evaluation showed. In this work, existing algorithms are benchmarked on a new, more realistic database. This database consists of two different data sets, from which morphs were created using four different morphing algorithms. In addition, the database contains four different post-processings (including print-scan transformation and JPEG2000 compression). Further, a new morphing attack detection method based on a fusion of different configurations of Multi-scale Block Local Binary Patterns (MB-LBP) on an image divided into multiple cells is presented. The proposed score-level fusion of a maximum number of 18 different configurations is shown to significantly improve the robustness of the resulting morphing attack detection scheme, yielding an average performance between 2.26% and 8.52% in terms of Detection Equal Error Rate (D-EER), depending on the applied post-processing.

Weakly supervised single image dehazing

Single image dehazing is a critical image pre-processing step for many practical vision systems. Most existing dehazing methods solve this problem utilizing various of hand-crafted priors or by supervised training on the synthetic hazy image information (such as haze-free image, transmission map and atmospheric light). However, the assumptions on the hand-crafted priors are easily violated and collecting realistic transmission map and atmospheric light are unpractical. In this paper, we propose a novel weakly supervised network based on the multi-level multi-scale block. The proposed network reduces the constraint on the training data and automatically estimates the transmission map and the atmospheric light as well as the intermediate haze-free image without using any realistic transmission map and atmospheric light as supervision. Moreover, the estimated intermediate haze-free image helps to generate accurate transmission map and atmospheric light by embedding the physical-model, which presents reliable restoration of the final haze-free image. In particular, our network also can be trained on the real-world dataset to fine-tune the model and the fine-tuning operation improves the dehazing performance on the real-world dataset. Quantitative and qualitative experimental results demonstrate the proposed method performs on par with the supervised methods.

Ghost Removal via Channel Attention in Exposure Fusion

High dynamic range (HDR) imaging is to reconstruct high-quality images with a broad range of illuminations from a set of differently exposed images. Some existing algorithms align the input images before merging them into an HDR image, but artifacts of the registration appear due to misalignment. Recent works try to remove the ghosts by detecting motion region or skipping the registered process, however, the result still suffers from ghost artifacts for scenes with significant motions. In this paper, we propose a novel Multi-scale Channel Attention guided Network (MCANet) to address the ghosting problem. We use multi-scale blocks consisting of dilated convolution layers to extract informative features. The channel attention blocks suppress undesired components and guide the network to refine features to make full use of feature maps. The proposed MCANet recovers the occluded or saturated details and reduces artifacts due to misalignment. Experiments show that the proposed MCANet can achieve state-of-the-art quantitative and qualitative results.

A Triple Residual Multiscale Fully Convolutional Network Model for Multimodal Infant Brain MRI Segmentation

The accurate segmentation of infant brain MR image into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is very important for early studying of brain growing patterns and morphological changes in neurodevelopmental disorders. Because of inherent myelination and maturation process, the WM and GM of babies (between 6 and 9 months of age) exhibit similar intensity levels in both T1-weighted (T1w) and T2-weighted (T2w) MR images in the isointense phase, which makes brain tissue segmentation very difficult. We propose a deep network architecture based on U-Net, called Triple Residual Multiscale Fully Convolutional Network (TRMFCN), whose structure exists three gates of input and inserts two blocks: residual multiscale block and concatenate block. We solved some difficulties and completed the segmentation task with the model. Our model outperforms the U-Net and some cutting-edge deep networks based on U-Net in evaluation of WM, GM and CSF. The data set we used for training and testing comes from iSeg-2017 challenge (http://iseg2017.web.unc.edu).

WGAN-E: A Generative Adversarial Networks for Facial Feature Security

Artificial intelligence technology plays an increasingly important role in human life. For example, distinguishing different people is an essential capability of many intelligent systems. To achieve this, one possible technical means is to perceive and recognize people by optical imaging of faces, so-called face recognition technology. After decades of research and development, especially the emergence of deep learning technology in recent years, face recognition has made great progress with more and more applications in the fields of security, finance, education, social security, etc. The field of computer vision has become one of the most successful branch areas. With the wide application of biometrics technology, bio-encryption technology came into being. Aiming at the problems of classical hash algorithm and face hashing algorithm based on Multiscale Block Local Binary Pattern (MB-LBP) feature improvement, this paper proposes a method based on Generative Adversarial Networks (GAN) to encrypt face features. This work uses Wasserstein Generative Adversarial Networks Encryption (WGAN-E) to encrypt facial features. Because the encryption process is an irreversible one-way process, it protects facial features well. Compared with the traditional face hashing algorithm, the experimental results show that the face feature encryption algorithm has better confidentiality.

Multiscale block averaging for QED in d = 3

We continue the study of the ultraviolet problem for quantum electrodynamics in d = 3. The model is defined on a fine toroidal lattice, and we seek control as the lattice spacing goes to zero. The problem is analyzed using Balaban’s formulation of the renormalization group. This involves a sequence of transformations consisting of a split into large and small field regions, then block averaging, and then scaling. The effective actions generated by this method depend strongly on certain multi-scale propagators and minimizers. The study of these objects both for fermions and for gauge fields is the content of this paper. Earlier work on the subject is reviewed. In addition for fermions, a polymer expansion is obtained for the determinants of the fermion propagators. For the gauge field, a detailed local regularity result is obtained for the minimizers.

Remaining useful life prediction using multi-scale deep convolutional neural network

Accurate and reliable remaining useful life (RUL) assessment result provides decision-makers valuable information to take suitable maintenance strategy to maximize the equipment usage and avoid costly failure. The conventional RUL prediction methods include model-based and data-driven. However, with the rapid development of modern industries, the physical model is becoming less capable of describing sophisticated systems, and the traditional data-driven methods have limited ability to learn sophisticated features. To overcome these problems, a multi-scale deep convolutional neural network (MS-DCNN) which have powerful feature extraction capability due to its multi-scale structure is proposed in this paper. This network constructs a direct relationship between Condition Monitoring (CM) data and ground-RUL without using any prior information. The MS-DCNN has three multi-scale blocks (MS-BLOCKs), where three different sizes of convolution operations are put on each block in parallel. This structure improves the network’s ability to learn complex features by extracting features of different scales. The developed algorithm includes three stages: data pre-processing, model training, and RUL prediction. After the min–max normalization pre-processing, the data is sent to the MS-DCNN network for parameter training directly, and the associated RUL value can be estimated base on the learned representations. Regularization helps to improve prediction accuracy and alleviate the overfitting problem. We evaluate the method on the available modular aero-propulsion system simulation data (C-MAPSS dataset) from NASA. The results show that the proposed method achieves good prognostics performance compared with other network architectures and state-of-the-art methods. RUL prediction result is obtained precisely without increasing the calculation burden.

Texture classification using block intensity and gradient difference (BIGD) descriptor

In this paper, we present an efficient and distinctive local descriptor, namely block intensity and gradient difference (BIGD). In an image patch, we randomly sample multi-scale block pairs and utilize the intensity and gradient differences of pairwise blocks to construct the local BIGD descriptor. The random sampling strategy and the multi-scale framework help BIGD descriptors capture the distinctive patterns of patches at different orientations and spatial granularity levels. We use vectors of locally aggregated descriptors (VLAD) or improved Fisher vector (IFV) to encode local BIGD descriptors into a full image descriptor, which is then fed into a linear support vector machine (SVM) classifier for texture classification. We compare the proposed descriptor with typical and state-of-the-art ones by evaluating their classification performance on five public texture data sets including Brodatz, CUReT, KTH-TIPS, and KTH-TIPS-2a and -2b. Experimental results show that the proposed BIGD descriptor with stronger discriminative power yields 0.12%∼6.43% higher classification accuracy than the state-of-the-art texture descriptor, dense microblock difference (DMD).

Adaptive Multiscale Block Compressed Sensing of Images based on Gray Level Co- Occurrence Matrix

In conventional block compressed sensing (BCS), the images are divided into small fixed-size blocks sampled at the same sub-rate. The sparsities and high-frequency components of the images are ignored, and the reconstruction qualities of the complex texture images are poor. An adaptive multiscale variant of the block compressed sensing was proposed to reconstruct the texture details of the images. The texture features of the images were obtained from the high-frequency components by the three-level wavelet transform and analyzed on the basis of the gray level co-occurrence matrix. A mathematical model was established to adjust the block sizes of the images automatically and allocate the limited sampling resource adaptively. The smoothed projected Landweber (SPL) was utilized to reconstruct the images. The accuracy of the proposed algorithm was verified by the simulation experiments. Results demonstrate that the texture details of the reconstructed images are abundant. The image edges are also clear, and the blocking artifacts are effectively eliminated. The reconstruction qualities of images, especially the partial images, are considerably improved at different sub-sampling rates. The proposed algorithm achieves a 2.42–3.3 dB gain in reconstruction PSNR for the Barbara image over the original BCS-SPL at a sub-sampling rate of 0.3. No remarkable differences are noted between the reconstructed and original texture blocks in visual sensation. The proposed algorithm provides evidence for the compression and reconstruction of the images with complex texture details.

Classification Epileptic Seizures in EEG Using Time-Frequency Image and Block Texture Features

With the rapid development in technology, computer aided detection or diagnosis has become an indispensable part of the medical industry. Automatic detection of epileptic events is one of the important subjects that have aroused wide interest from more and more investigators. This paper proposes a new model in classification of multi-category electroencephalogram (EEG) signals using time-frequency image and block texture features. The one-dimensional EEG is first mapped to time-frequency domain by means of short-time Fourier transform (STFT), which is adapted to obtain a two-dimensional time-frequency image (2D-TFI). With the idea of multi-scale blocking, the obtained phase images and amplitude images are divided into several sub-blocks corresponding to different frequency ranges and time periods. Then the texture features are calculated to describe the behaviour of EEG signals. Particularly, a novel quadratic feature selection method based on kernel entropy component analysis (KECA) and Kruskal-Wallis test (KW) has been proposed for dimension reduction, by which the features that contained most distinctive information were provided. Eventually, the optimal KECA-based features are fed to support vector machine (SVM) for deciding the class of corresponding EEG. The proposed model is found to achieve at least 99.30% accuracy, 98.0% sensitivity and 100% specificity for each of the eight clinical problems. Our scheme is proven to be effective for seizure detection, which can help doctors optimize the diagnosis workflow, reduce workload and improve detection precision.

A New Multi-Scale Convolutional Model Based on Multiple Attention for Image Classification

Computer vision systems are insensitive to the scale of objects in natural scenes, so it is important to study the multi-scale representation of features. Res2Net implements hierarchical multi-scale convolution in residual blocks, but its random grouping method affects the robustness and intuitive interpretability of the network. We propose a new multi-scale convolution model based on multiple attention. It introduces the attention mechanism into the structure of a Res2-block to better guide feature expression. First, we adopt channel attention to score channels and sort them in descending order of the feature’s importance (Channels-Sort). The sorted residual blocks are grouped and intra-block hierarchically convolved to form a single attention and multi-scale block (AMS-block). Then, we implement channel attention on the residual small blocks to constitute a dual attention and multi-scale block (DAMS-block). Introducing spatial attention before sorting the channels to form multi-attention multi-scale blocks(MAMS-block). A MAMS-convolutional neural network (CNN) is a series of multiple MAMS-blocks. It enables significant information to be expressed at more levels, and can also be easily grafted into different convolutional structures. Limited by hardware conditions, we only prove the validity of the proposed ideas through convolutional networks of the same magnitude. The experimental results show that the convolution model with an attention mechanism and multi-scale features is superior in image classification.

Identifying protein-protein interface via a novel multi-scale local sequence and structural representation

BackgroundProtein-protein interaction plays a key role in a multitude of biological processes, such as signal transduction, de novo drug design, immune responses, and enzymatic activities. Gaining insights of various binding abilities can deepen our understanding of the interaction. It is of great interest to understand how proteins in a complex interact with each other. Many efficient methods have been developed for identifying protein-protein interface.ResultsIn this paper, we obtain the local information on protein-protein interface, through multi-scale local average block and hexagon structure construction. Given a pair of proteins, we use a trained support vector regression (SVR) model to select best configurations. On Benchmark v4.0, our method achieves average Irmsd value of 3.28Å and overall Fnat value of 63%, which improves upon Irmsd of 3.89Å and Fnat of 49% for ZRANK, and Irmsd of 3.99Å and Fnat of 46% for ClusPro. On CAPRI targets, our method achieves average Irmsd value of 3.45Å and overall Fnat value of 46%, which improves upon Irmsd of 4.18Å and Fnat of 40% for ZRANK, and Irmsd of 5.12Å and Fnat of 32% for ClusPro. The success rates by our method, FRODOCK 2.0, InterEvDock and SnapDock on Benchmark v4.0 are 41.5%, 29.0%, 29.4% and 37.0%, respectively.ConclusionExperiments show that our method performs better than some state-of-the-art methods, based on the prediction quality improved in terms of CAPRI evaluation criteria. All these results demonstrate that our method is a valuable technological tool for identifying protein-protein interface.

Nickel foam surface defect detection based on spatial-frequency multi-scale MB-LBP

According to the nickel foam surface defect images with the typical characteristics of complex geometry and texture distribution, a nickel foam surface defect detection method based on spatial-frequency multi-scale block local binary pattern is proposed. First, nonsubsampled contourlet is used to carry out foam nickel image multi-scale decomposition, and therefore, low-frequency sub-band images and high-frequency sub-band images are obtained. The multi-scale block local binary pattern is then used to extract the feature histogram vectors of each block region of low- and high-frequency sub-bands, and the histogram feature vectors of the whole image after cascade are formed. The kernel principal component analysis and support vector machine are adopted to reduce the dimension of the feature histogram vectors and used for the defect classification. Experimental results show that the proposed method of feature extraction can extract more detailed texture information, and the average recognition rate reaches to 90%, which meets an enterprise’s needs.

A local multiple patterns feature descriptor for face recognition

Human perception of a signal depends on the ratio of the change of stimulus to the stimulus itself while the change of stimulus to the stimulus itself is usually ignored in hand-crafted feature descriptors. However, it is important for extracting discriminant feature. To address this problem, we firstly develop a local multiple patterns (LMP) feature descriptor based on the Weber's law for feature extraction and face recognition. In LMP, (1) the Weber's ratio is modified to contain the change direction, and thus the modified Weber's ratio is quantized into several intervals to generate multiple feature maps for describing different changes; (2) LMP concatenates the histograms of the non-overlapping regions of the feature maps for image representation. Secondly, since LMP could only capture small-scale structures, a multi-scale block LMP (MB-LMP) is presented to generate more discriminative and robust visual feature. Thus, MB-LMP could capture both the small-scale and large-scale structures. In implementation, MB-LMP is computationally efficient using integral image. In experiments, LMP and MB-LMP are evaluated on four public datasets for face recognition. The experimental results demonstrate the promise of the proposed LMP and MB-LMP descriptors with desirable efficiency.

Weighted-fusion feature of MB-LBPUH and HOG for facial expression recognition

Obtaining a useful and discriminative feature for facial expression recognition (FER) is a hot research topic in computer vision. In this paper, we propose a novel facial expression representation for FER. Firstly, we select the appropriate parameter of multi-scale block local binary pattern uniform histogram (MB-LBPUH) operator to filter the facial images for representing the holistic structural features. Then, normalizing the filtered images into a uniform basis reduces the computational complexity and remains the full information. An MB-LBPUH feature and a HOG feature are concatenated to fuse a new feature representation for characterizing facial expressions. At the same time, weighting the MB-LBPUH feature can remove the data unbalance from a fusion feature. The weighted-fusion feature reflects not only global facial expressions structure patterns but also characterizes local expression texture appearance and shape. Finally, we utilize principal component analysis for dimensionality reduction and employ support vector machine to classification. Experimental results demonstrate that the proposed algorithm exhibits superior performance compared with the existing algorithms on JAFFE, CK+, and BU-3DFE datasets.

Frequency hopping radio individual identification based on energy spectrum blended subtle characteristics

For the problem of low recognition rate and large computational complexity of frequency hopping radio stations, a blended subtle characteristic extraction method based on energy spectrum is proposed. Considering that the time-frequency energy distribution is widely used in individual feature extraction, the time-frequency energy spectrum of the frequency hopping signal is obtained by the sparse reconstruction method firstly. Then the fractal theory is used to extract the feature vector set including time-frequency energy spectrum Rayleigh entropy, multi-fractal dimension and difference box dimension with multi-scale block extraction. Finally, the support vector machine classifier is used to train, classify and identify the feature set to realize the individual identification of the frequency hopping station. The recognition performance of the proposed method and the other two methods are compared and verified through the frequency hopping signals of four stations. The experimental results show that compared with the other two methods, the proposed method has higher recognition rate under low SNR and a small number of training samples.

Conversion of the geological model (fine-mesh) to a dynamic (coarse-mesh) hydrocarbon model with the nature approach in the simulation of thermal recovery in a fractured reservoir

The operation and proper management of reservoirs requires the prediction of reservoir performance. This prediction is generally done by computer simulation. Since simulation software generates static models in the number of millions and even billions, dynamic simulation on these models is difficult and sometimes impossible; therefore, multi-scale block generation is necessary. In this study, we introduce a methodology that was inspired by Earth’s pattern to generate multi-scale grids on a multi-phase, heterogeneous reservoir, where the enhanced oil recovery process is steam injection. From a reservoir engineer’s point of view, Earth is equivalent to a multi-scale grid model that could be a pattern for multi-scale grid generation in a hydrocarbon reservoir to minimize central processing unit time for dynamic simulation. The principle of multi-scale grid generation in hydrocarbon reservoirs is as follows: regions with a high Darcy’s velocity should be fine-scale and other segments with low intensity of heterogeneity regions could resize into the up-scale. The intersection of latitude and longitude lines means that Earth’s model could be a practical pattern for dynamic simulation. This intersection creates segments with fine-size blocks, like the South and North Poles, that are equal to the injection and production wells in the reservoir model and other segments that have different intensities of coarse-size blocks. Earth’s magnetic field, which enters from South and exits from the North Pole, leads us to could consider Earth as a hydrocarbon model. The results of the multi-scale grid generation method, which was inspired by Earth’s pattern, were compared with the fine-mesh (geological) model; the results show that the proposed method predicts with high accuracy and less run-time compared with the fine-mesh model.

Traffic Density Recognition Based on Image Global Texture Feature

Traffic state recognitions can provide a strategic support for control and management of urban traffic, which is crucial to ease traffic congestion, reduce road accidents, and ensure road traffic efficiency. This paper proposes an effective traffic density estimation method based on image processing. In the beginning, a whole image is divided into several cells, and then a region of interest (ROI) is extracted based on calculating varieties of pixel values in a temporal sequence of each cell. Then a texture feature descriptor, a histogram of multi-scale block local binary pattern (HMBLBP) is proposed for local feature representation. The HMBLBP of all cells in the ROI are concatenated as a global feature. Furthermore, principle component analysis is performed for dimensionality reduction to save computational cost. At last, the method proposed is tested with two datasets captured from real-world traffic scenarios. By using the support vector machine (SVM) classifier, traffic states are classified into heavy, medium and light densities. Reliable performances are shown in the experimental tests.

多尺度分块的自适应采样率压缩感知算法

多尺度分块的自适应采样率压缩感知算法

Multiscale Block Fusion Object Detection Method for Large-Scale High-Resolution Remote Sensing Imagery

Object detection in high-spatial-resolution remote sensing images (HSRIs) is an import part of the automatic extraction and understanding of image information in high-resolution earth observation systems. Regarding how to achieve optimal block object detection for large-scale HSRIs, this paper proposes a multiscale block fusion object detection method for large-scale HSRIs. First, the objects in large-scale HSRIs are detected using different block scales, and the average precision (AP) of the different object detection results is counted at different block scales. Then, according to the statistical information, the image block scales corresponding to the optimal AP value of the different objects are obtained. Finally, a soft non-maximum suppression algorithm is used to fuse the image block scale detection results corresponding to the optimal AP values of the different objects, to obtain the object detection results of the large-scale HSRIs. The experimental results confirm that the proposed method outperforms all other single-scale image block detection methods and provides acceptable object detection results in large-scale HSRIs.