Spatial Pyramid Matching Research Articles

An improved prairie dog optimization algorithm integrating multiple strategies and its application

Aiming at the problems in prairie dog optimization (PDO), such as uneven population distribution at initialization, slow convergence, imbalance between global exploration and local exploitation, and the tendency to get trapped in the local optimum, this study proposes an Improved prairie dog optimisation algorithm integrating multiple strategies (IMSPDO). Firstly, the population is initialized using spatial pyramid matching (SPM) chaotic mapping combined with improved random opposition-based learning (IROL) to solve the problems of uneven distribution and poor diversity of the population. Secondly, the prey escapes energy formula mentioned in the harris hawks optimization (HHO) is introduced to achieve the smooth transition between the exploration phase and the exploitation phase, balancing the algorithm’s global exploration capability and local exploitation capability. Additionally, the idea of the particle swarm optimization (PSO) is applied to enhance the global optimization capability of the algorithm. Finally, the ideas of simulated annealing (SA), polynomial mutation and Cauchy mutation are also introduced to improve the ability that individuals to jump out of the local optimum. The performance of the improved algorithm is verified on a set of 21 classical benchmark functions and 8 CEC2020 test functions. The proposed IMSPDO is also evaluated against original PDO, and six other commonly used algorithms. The result of the Wilcoxon rank-sum test shows that there is a significant difference between the selected algorithms and IMSPDO. Furthermore, 3 engineering examples are used to further test the superiority of IMSPDO in dealing with real-world problems.

Efficient and Accurate Image Classification via Spatial Pyramid Matching and SURF Sparse Coding

Sparsely coded signal space representations do well in feature quantization. Instead of using standard vector quantization, the suggested method uses selective sparse coding to assemble the most important features of the appearance descriptors of nearby image patches. Inadequate coding also enables adjacent max pooling on some spatial scales, which, unlike the setup of average pooling in a histogram, links interpretation with scale invariance. The acquired visual illustration is the key contribution of this research; it performs outperform with linear-SVM, improves the model training's, which in turn speeds up testing with improves accuracy. The efficacy of the method we have employed has been substantiated through a series of experiments conducted on diverse datasets. Since top-performing image classification systems heavily rely on nonlinear SPM in mean of vector quantization, the trustworthy recommended linear SPM greatly increases the use of larger sets of training data. The method given herein deduces that the sparse coding of SURF feature’s function hampered a more comprehensive local appearance descriptor for general-purpose image processing. Experiments and comparisons are conducted on standard datasets such as Caltech-101, FTVL, and Corel-1000, using state-of-the-art techniques and descriptors. When compared over several other image categories and descriptors, the method provided here comes out on top.

Design of automatic style classification system of visual art works based on image processing

Abstract The emergence of a vast array of visual artworks meets the growing public demand for cultural literacy while simultaneously presenting challenges in the understanding and selection of appropriate artistic styles. Leveraging advancements in image processing technology, this study successfully automates the classification of 158,652 visual art images by categorizing them into distinct artistic styles. A novel approach combining the Scale-Invariant Feature Transform (SIFT) feature extraction algorithm with the spatial pyramid matching technique is introduced. This paper details the development and evaluation of a classification system tailored for the automatic categorization of visual artworks. Comparative testing results demonstrate that this innovative combination surpasses the traditional SIFT algorithm in both execution speed and classification accuracy. Specifically, the Support Vector Machine (SVM) based classification system, utilizing the enhanced SIFT method, achieves an accuracy rate exceeding 90% in identifying five distinct styles, including PHPS, NOS, IS, TF, and AAS. The implementation of this automatic classification system proves to be a practical tool in aiding individuals to select the visual artwork style that best suits their preferences.

Human Behavior Recognition via Hierarchical Patches Descriptor and Approximate Locality-Constrained Linear Coding.

Human behavior recognition technology is widely adopted in intelligent surveillance, human-machine interaction, video retrieval, and ambient intelligence applications. To achieve efficient and accurate human behavior recognition, a unique approach based on the hierarchical patches descriptor (HPD) and approximate locality-constrained linear coding (ALLC) algorithm is proposed. The HPD is a detailed local feature description, and ALLC is a fast coding method, which makes it more computationally efficient than some competitive feature-coding methods. Firstly, energy image species were calculated to describe human behavior in a global manner. Secondly, an HPD was constructed to describe human behaviors in detail through the spatial pyramid matching method. Finally, ALLC was employed to encode the patches of each level, and a feature coding with good structural characteristics and local sparsity smoothness was obtained for recognition. The recognition experimental results on both Weizmann and DHA datasets demonstrated that the accuracy of five energy image species combined with HPD and ALLC was relatively high, scoring 100% in motion history image (MHI), 98.77% in motion energy image (MEI), 93.28% in average motion energy image (AMEI), 94.68% in enhanced motion energy image (EMEI), and 95.62% in motion entropy image (MEnI).

Research on Image Classification and Key Technologies Based on 3D Feature Extraction Algorithm

Image classification and recognition has a very wide range of applications in computer vision, which involves many fields, such as image retrieval, image analysis, and robot positioning. Especially with the rise of brain science and cognitive science research, as well as the increasing diversification of imaging means, three-dimensional image data mainly based on magnetic resonance image plays an increasingly important role in image classification and recognition, especially in medical image classification and recognition. However, due to the high dimensional characteristics of human magnetic resonance images, human readability is reduced. Therefore, classification and recognition of 3-dimensional images is still a challenge. In order to better extract local features from images and effectively use their spatial information, this paper improved the “feature bag” and “spatial pyramid matching” algorithms on the basis of 3D feature extraction algorithm and proposed an image classification framework based on 3D feature extraction algorithm. Firstly, the multiresolution “3D spatial pyramid” algorithm, the multiscale image segmentation and image representation method, and the SVM classifier and feature fusion method are described. Secondly, the gender information contained in the magnetic resonance images is classified and recognized on the three databases selected in the experiment. Experimental results show that this method can effectively utilize the spatial information of three-dimensional images and achieve satisfactory results in the classification and recognition of human magnetic resonance images.

Pitch Evaluation of Matouqin Chamber Music Performance Based on Artificial Neural Network

In order to study the pitch evaluation of Matouqin chamber music performance based on artificial neural network, this paper puts forward the relevant theories in the fields of human ear auditory perception system, auditory psychology, music theory knowledge, and pattern recognition. This paper extracts the auditory image features of chords and then establishes a sparse representation classifier model for chord recognition and classification. Scale-invariant feature transformation (SIFT) and spatial pyramid matching (SPM) are used to extract the detailed features of chord auditory images. The experimental results show that the highest correct recognition rate of the chord recognition algorithm based on the auditory image proposed in this paper is 76.2%, which is 20.4% higher than that of MFCC feature based on human auditory characteristics.

Sparse Weighting for Pyramid Pooling-Based SAR Image Target Recognition

In this study, a novel feature learning method for synthetic aperture radar (SAR) image automatic target recognition is presented. It is based on spatial pyramid matching (SPM), which represents an image by concatenating the pooling feature vectors that are obtained from different resolution sub-regions. This method exploits the dependability of obtaining the weighted pooling features generated from SPM sub-regions. The dependability is determined by the residuals obtained from sparse representation. This method aims at enhancing the weights of the pooling features generated in the sub-regions located in the target and suppressing the weights of the background. The feature representation for SAR image target recognition is discriminative and robust to speckle noise and background clutter. Experiments performed on the Moving and Stationary Target Acquisition and Recognition public dataset prove the advantageous performance of the presented algorithm over several state-of-the-art methods.

Local-Tetra-Patterns for Face Recognition Encoded on Spatial Pyramid Matching

Face recognition is a big challenge in the research field with a lot of problems like misalignment, illumination changes, pose variations, occlusion, and expressions. Providing a single solution to solve all these problems at a time is a challenging task. We have put some effort to provide a solution to solving all these issues by introducing a face recognition model based on local tetra patterns and spatial pyramid matching. The technique is based on a procedure where the input image is passed through an algorithm that extracts local features by using spatial pyramid matching and max-pooling. Finally, the input image is recognized using a robust kernel representation method using extracted features. The qualitative and quantitative analysis of the proposed method is carried on benchmark image datasets. Experimental results showed that the proposed method performs better in terms of standard performance evaluation parameters as compared to state-of-the-art methods on AR, ORL, LFW, and FERET face recognition datasets.

Improving Gray World Algorithm Guided by Scene Semantics

Gray world algorithm typically shows poor illuminant estimation performance due to scene variations of natural images. To address the above problem, an improved gray world algorithm guided by scene semantics is proposed. Firstly, dense-SIFT descriptors are calculated using grayscale images to avoid the interference of color-biased images, followed by the bag of words (BoW) model to generate unordered visual vocabulary. Secondly, based on the average brightness of these visual words, word frequency histograms are constructed weighted by brightness using the spatial pyramid matching (SPM) algorithm. Thirdly, the scene semantics similarity between images employing the histogram cross kernel function, and retrieve a set of candidate images that similar to the test image is calculated. Finally, after removing the outlier images using the isolated forest algorithm, the pixel statistics distributions of all remaining images are used to adaptively infer and update the fixed assumption of the gray world algorithm. The improved gray world algorithm implements color constancy through illuminant estimation. Experimental results in three publicly available color constancy datasets (ColorChecker, Cube+ and NUS) show that the proposed algorithm outperforms comparable improved gray world algorithms in single camera tests (nearly 20% angular error improvement), at the same time, achieves optimal performance in cross-camera tests.

Sparse coded spatial pyramid matching and multi-kernel integrated SVM for non-linear scene classification

AbstractSupport vector machine (SVM) techniques and deep learning have been prevalent in object classification for many years. However, deep learning is computation-intensive and can require a long training time. SVM is significantly faster than Convolution Neural Network (CNN). However, the SVM has limited its applications in the mid-size dataset as it requires proper tuning. Recently the parameterization of multiple kernels has shown greater flexibility in the characterization of the dataset. Therefore, this paper proposes a sparse coded multi-scale approach to reduce training complexity and tuning of SVM using a non-linear fusion of kernels for large class natural scene classification. The optimum features are obtained by parameterizing the dictionary, Scale Invariant Feature Transform (SIFT) parameters, and fusion of multiple kernels. Experiments were conducted on a large dataset to examine the multi-kernel space capability to find distinct features for better classification. The proposed approach founds to be promising than the linear multi-kernel SVM approaches achieving 91.12 % maximum accuracy.

Bag of shape descriptor using unsupervised deep learning for non-rigid shape recognition

Highly discriminative feature expression for non-rigid shape recognition is an important and challenging task, which requires both abstract and robust shape descriptors. However, the majority of existing low-level descriptors are designed via hand-crafted, which are sensitive to local changes and larger deformation. To address this issue, this paper proposes a bag of shape descriptor based on unsupervised deep learning and Bag of Words (BoW) for shape recognition. Different from existing pipelines, our method is specially designed to learn high-level and hierarchical shape features from multi-scale context structures. It effectively overcomes obstacles, such as irregular topology, orientation ambiguity, and rigid or non-rigid transformation in the hierarchical learning of contour fragments. Specifically, by adopting an improved decomposing strategy, the shape can be decomposed to a series of valuable contour fragments, results in local to global feature learning. An unsupervised learning framework is also applied to the contour fragment for its feature expression based on the context structure and SSAE (Stack Sparse Auto Encode). In the process of shape representation, a high-level shape dictionary is learned by K-clustering to achieve discriminative feature coding. In addition, to achieve a compact and simplified shape representation, SPM (Spatial Pyramid Matching) is adopted by max-pooling, which effectively incorporates spatial layout information of the given shape. The experiments demonstrate that the proposed method achieves state-of-the-art performance on several public shape datasets comparing with the latest approaches. Our method also obtains high performance under the noisy and occlusion condition.

Evaluation of image partitioning strategies for preserving spatial information of cross-sectional micrographs in automated wood recognition of Fagaceae

Although wood cross sections contain spatiotemporal information regarding tree growth, computer vision-based wood identification studies have traditionally favored disordered image representations that do not take such information into account. This paper describes image partitioning strategies that preserve the spatial information of wood cross-sectional images. Three partitioning strategies are designed, namely grid partitioning based on spatial pyramid matching and its variants, radial and tangential partitioning, and their recognition performance is evaluated for the Fagaceae micrograph dataset. The grid and radial partitioning strategies achieve better recognition performance than the bag-of-features model that constitutes their underlying framework. Radial partitioning, which is a strategy for preserving spatial information from pith to bark, further improves the performance, especially for radial-porous species. The Pearson correlation and autocorrelation coefficients produced from radially partitioned sub-images have the potential to be used as auxiliaries in the construction of multi-feature datasets. The contribution of image partitioning strategies is found to be limited to species recognition and is unremarkable at the genus level.

Clustering Crowdsourced Test Reports of Mobile Applications Using Image Understanding

Crowdsourced testing has been widely used to improve software quality as it can detect various bugs and simulate real usage scenarios. Crowdsourced workers perform tasks on crowdsourcing platforms and present their experiences as test reports, which naturally generates an overwhelming number of test reports. Therefore, inspecting these reports becomes a time-consuming yet inevitable task. In recent years, many text-based prioritization and clustering techniques have been proposed to address this challenge. However, in mobile testing, test reports often consist of only short test descriptions but rich screenshots. Compared with the uncertainty of textual information, well-defined screenshots can often adequately express the mobile application's activity views. In this paper, by employing image-understanding techniques, we propose an approach for clustering crowdsourced test reports of mobile applications based on both textual and image features to assist the inspection procedure. We employ Spatial Pyramid Matching (SPM) to measure the similarity of the screenshots and use the natural-language-processing techniques to compute the textual distance of test reports. To validate our approach, we conducted an experiment on 6 industrial crowdsourced projects that contain more than 1600 test reports and 1400 screenshots. The results show that our approach is capable of outperforming the baselines by up to 37% regarding the APFD metric. Further, we analyze the parameter sensitivity of our approach and discuss the settings for different application scenarios.

A NOVEL SELF-TAUGHT LEARNING FRAMEWORK USING SPATIAL PYRAMID MATCHING FOR SCENE CLASSIFICATION

Abstract. Remote sensing earth observation images have a wide range of applications in areas like urban planning, agriculture, environment monitoring, etc. While the industrial world benefits from availability of high resolution earth observation images since recent years, interpreting such images has become more challenging than ever. Among many machine learning based methods that have worked out successfully in remote sensing scene classification, spatial pyramid matching using sparse coding (ScSPM) is a classical model that has achieved promising classification accuracy on many benchmark data sets. ScSPM is a three-stage algorithm, composed of dictionary learning, sparse representation and classification. It is generally believed that in the dictionary learning stage, although unsupervised, one should use the same data set as classification stage to get good results. However, recent studies in transfer learning suggest that it might be a better strategy to train the dictionary on a larger data set different from the one to classify.In our work, we propose an algorithm that combines ScSPM with self-taught learning, a transfer learning framework that trains a dictionary on an unlabeled data set and uses it for multiple classification tasks. In the experiments, we learn the dictionary on Caltech-101 data set, and classify two remote sensing scene image data sets: UC Merced LandUse data set and Changping data set. Experimental results show that the classification accuracy of proposed method is compatible to that of ScSPM. Our work thus provides a new way to reduce resource cost in learning a remote sensing scene image classifier.

Video retrieval using salient foreground region of motion vector based extracted keyframes and spatial pyramid matching

Despite enormous research efforts devoted by the research community to effectively and precisely perform video matching and retrieval among heterogeneous videos from large-scale video repositories still remains a complex and most challenging task. In order to address this complex challenge, a content based video retrieval technique is required, which can exploit the visual content of the videos for effective retrieval from the videos repositories. In our proposed method, we introduce a computer assisted video retrieval technique which can retrieve the visually similar videos stored in the repositories. To accomplish this task, video summarization based on motion vector is employed to select keyframes based on similar segments. To estimate the video content, salient foreground extraction is executed, and matching based on the spatial pyramid is employed for matching the keyframe features of query video with videos in the repositories. The contribution of the former process has two major sections for superior saliency map generation. Firstly, it heuristically integrates the regional property, contrast, and foreground descriptors together. Secondly, it introduces a new feature vector to characterize the foreground as an object descriptor, while the latter process is the extension of orderless bag-of-features representation, which has significant performance with respect to scene categorization. The video retrieval performance is compared with standard state-of-the-art techniques using real-time datasets. Experimental and usability studies provide satisfactory results for video retrieval based on evaluation metrics such as video sampling error, fidelity, precision, and recall.

Detection of heterogeneity in multi-spectral transmission image based on spatial pyramid matching model and deep learning

The absorption and scattering effects of light source during the transmission of biological tissues make it difficult to identify heterogeneity in multi-spectral images. This paper firstly proposes a combination method of modulation-demodulation-frame accumulation technique (MDFAT), spatial pyramid matching (SPM) model and deep learning to realize heterogeneous detection in multi-spectral images. Firstly, the acquisition experiment of phantom image is designed. Then, the high-quality multi-spectral images are obtained by the MDFAT and SPM model. Finally, the pseudo-color maps of high-quality multi-spectral images fusion are served as the input of Faster-RCNN and Single Shot Multi-Box Detector (SSD) network models to realize heterogeneous detection. The results show that Faster-RCNN and SSD both have good detection results. Among them, Faster-RCNN model has the best detection effect on the images containing three types of heterogeneity, and the mean average precision (mAP) reaches 93.91%. SSD model has the most ideal detection effect for the images containing two and five types of heterogeneity, with mAP reaching 94.16% and 94.78% respectively. In conclusion, this paper has verified the feasibility of detecting heterogeneities in multi-spectral images through deep learning network (Faster-RCNN and SSD), which will promote the clinical application of multi-spectral transmission imaging in early screening of breast tumors.

RETRACTED ARTICLE: An efficient recognition system for preserving ancient historical documents of English characters

The clusters of historical documents are of great importance in terms of cultural and scientific. In order to access the documents, originality should be maintained. So conversion of digital form is highly required for recognition. While converting, those documents may be due to poor quality, overlapping of characters, complex background and so on. In this paper, an efficient system for recognizing English characters from degraded historical document images is proposed. Initially, modified Adaptive Thresholding based binarization process is performed to eliminate the noise content in the input image. The characters are segmented through the rectangular bounding box method. Then Local binary pattern (LBP) algorithm is enforced to extricate the features of each characters. Finally, Spatial Pyramid Matching (SPM) classifier is used for texture classification. HDLA 2011 dataset is employed to validate the proposed method. The proposed method achieves 94.6% recognition accuracy and 0.34 s computation time for Lucida Black-letter font. This method also outperforms better than the existing recognition techniques.

Intestinal Polyp Recognition Based on Salient Codebook Locality-Constrained Linear Coding with Annular Spatial Pyramid Matching

Gastrointestinal endoscopy has become a widely used technique to diagnose intestinal anomalies like polyp. However, the vast quantities of images produced in the detection process greatly increase the burden of clinicians and the misdiagnosis rate. To tackle this problem, a new computer-assisted detection system called Salient Codebook Locality-Constrained Linear Coding (SCLLC) with Annular Spatial Pyramid Matching (ASPM) is proposed in this paper to classify intestinal polyp images automatically. Firstly, SIFT features are extracted from images and k-means clustering method is utilized on the patch features to obtain the initial codebook. Secondly, the proposed SCLLC algorithm is employed to achieve the salient codebook and encode the features by emphasizing the salient basis vectors in feature coding process. Then, a max-pooling strategy of annular region segmentation based on Spatial Pyramid Matching is proposed to improve the effectiveness of processing for intestinal images. Finally, SVM classifier is developed to carry out polyp images classification tasks. The experimental results exhibit promising 94.10% accuracy, 91.20% sensitivity and 97.01% specificity. In addition, the 0.97 s detecting time indicates the feasibility of clinical application. Our proposed method can effectively improve the overall performance of intestinal polyp recognition by integrating the internal base vector correlation of codebook and considering the annular structure of the intestinal images.

Surgery Agreement Signature Authentication System for Mobile Health Care

Currently, the use of biometric systems is increasing following the increase in the non-face-to-face security transactions in the Health Care sector, where smart devices are extensively used. Additionally, hospital patients or their guardians had to sign every medical/surgery consent form with a pen. Currently, hospitals are attempting to digitalize the form to avoid its loss or delay to the operating room. Thus, this study proposes a surgery consent signature authentication system for the mobile health care system. Along with the vein or the fingerprint recognition technology, the smart electronic signature recognition technology is regarded as a new type of security solution for Mobile Health Care, which is a compound of Health Care and technology, or a smart contents and display technology. Thus, this study proposes a surgery agreement signature authentication system for Mobile Health Care while using the techniques, such as database segment units comparison in the cloud, Bag of Word, etc. The proposed system was implemented with Java language and developed in a way the reference signature stored in advance in a cloud database to be compared with the signature currently entered. For the comparison, the segment matching, spatial pyramid matching, and boundary matching techniques were used in addition to the Dynamic Time Warping (DTW) algorithm. Additionally, the system has been made lighter than the existing experimental products, so that it is easier to embed the system into a smart phone, tablet, or others. The Test Bed experiment result showed that the system operated flexibly.

New Approach to Improve the Classification Process of Multi-Class Objects

In recent years, several descriptors have been proposed in many image classification applications. Accelerated-KAZE (A-KAZE) is considered one of the descriptors that has shown high performance for feature extraction. A-KAZE uses a binary descriptor called modified-local difference binary, which is very efficient and invariant to changes in rotation and scale. This representation does not allow spatial information to be considered between objects in the image, which makes it possible to reduce the performances of the classification of the images. This article broaches a new approach to improve the performance of the A-KAZE descriptor for image classification. The authors first establish the connection between the A-KAZE descriptor and the bag of feature model. Then the Spatial Pyramid Matching (SPM) is adopted by exploiting the A-KAZE descriptor to reinforce its robustness by introducing spatial information. The results of the experiments on several datasets show that the A-KAZE descriptor with SPM gives very satisfactory results compared with other existing methods in the state of the art.