Codebook Model Research Articles

A deep learning algorithm for fast motion video sequences based on improved codebook model

A deep learning algorithm for fast motion video sequences based on improved codebook model

Evaluation and Simulation of Sports Balance Training and Testing Equipment Based on Medical Video Image Analysis

Aiming at the shortcomings of video analysis system in the current process of balance force training, a new video analysis system for balance force training is proposed to analyze the test effect of balance force training. This paper evaluates and tests the function and utility of balance testing equipment by using the methods of sports training, sports biomechanics, mechanical design, sensors, video image analysis and computer-related research, combined with the latest research results of physical training at home and abroad. Firstly, several representative algorithms are selected in the frame difference method and background difference method. Then, these methods are tested according to the typical scene conditions, and the experimental results are used to compare and analyze the algorithms qualitatively and quantitatively, and the paper proposes the moving object detection and tracking based on GVF-Snake image model. The experimental results show that our proposed model has good detection effect compared with the background difference method based on Vibe algorithm, codebook model, Mixed Gauss model, adaptive Mixed Gauss model, median filter and adaptive background modeling. The system can accurately analyze the video and image information of sports training. The accuracy of key frame extraction is high, and the recall rate is high. It can be used to guide and evaluate balance force training. And the final evaluation results show the effectiveness of video image algorithm.

A hybrid codebook model for object categorization using two-way clustering based codebook generation method

Both the visual codebook and the codebook model are considered as two main parts of most object classification frameworks. In the original codebook model, each image descriptor is encoded using a single codebook obtained usually using a clustering approach. In this paper, we propose a hybrid codebook model for an object classification task. A simultaneous clustering approach is applied to image descriptors to generate two variant codebooks and used them separately to encode and represent an image through a patch-based codebook model and a feature-based codebook model respectively. The proposed codebook model has been tested on the Caltech-101 dataset. Experimental results demonstrate state-of-the-art performance compared to typical clustering-based codebook model.

Adaptive appearance modeling via hierarchical entropy analysis over multi-type features

The descriptiveness of visual models is crucial for many image processing applications, however, it is still challenging to adaptively formulate such models. This paper systematically advocates a generic and adaptive appearance modeling method. For object-specific instances in images, it can adaptively generate a descriptive codebook by exploring the maximum discriminability of multi-type features. The key idea is to define feature-independent information entropy as a unified criterion to measure different features in a common entropy space. Towards this goal, a hierarchical maximum entropy (HME) model is proposed to conduct multi-feature selection based on the random forest. Specifically, the improved random forest replaces space-specific expression “distance similarity” with the statistical concept “entropy”. Thus, the random forest could integrate the subspace clustering results from different feature spaces. Such integration can not only afford adaptive feature selection and cross-feature error control but also be robust to possible weak/inconsistent feature expressions. To effectively construct a class-specific appearance model, a sparse codebook model, consisting of a series of weak learners, is proposed to further explore the maximum discriminative subspaces of each object class. Finally, a maximum entropy model is proposed to formulate appearance model by optimizing the probabilistic distributions of all the codebook words’ response parameters. To verify the efficacy and effectiveness of the proposed model, it is applied to multi-class image classification. We conduct extensive experiments and make comprehensive evaluations w.r.t several state-of-the-art methods over PASCAL VOC 2007, VOC 2012, Caltech 101 and Caltech 256 datasets. All the results demonstrate the advantages of the our method in terms of precision, robustness, flexibility, and versatility.

Optimization of Three - Frame Difference Method and Improvement of Pedestrian Detection Code Book

Pedestrian detection is one of the most important applications of computer vision. In this paper, aiming at the problem of poor anti-interference ability of traditional codebook algorithm in the complex background and the cavity-prone problem of traditional frame difference method, a pedestrian detection algorithm integrating the optimized frame difference method and improved codebook is proposed. In this algorithm, a double code word structure is designed. An update mechanism is added to the code words in the codebook background modeling process and detection process, and improved double code word book algorithm based on YUV color space is designed to realize the real-time background update function. After that, bilateral filtering is added to the traditional three-frame difference method to further remove the noise. At the same time, the extracted rectangular region is used for filling operation and morphological processing to obtain the corresponding foreground image. Finally, the foreground image of codebook model and the foreground image of frame difference method are calculated with “and”, “different or” and “or” of pixels to obtain the precise region of moving target. The experimental results show that this algorithm combines the advantages of the two methods and abandons the disadvantages of each method in the detection prospect. It can accurately detect moving targets and has good robustness. The effectiveness of this algorithm is also verified by experiments.

A New Fault Diagnosis Method for a Diesel Engine Based on an Optimized Vibration Mel Frequency under Multiple Operation Conditions.

The diesel engine has been a significant component of large-scale mechanical systems for the intelligent manufacturing industry. Because of its complex structure and poor working environment, it has trouble effectively acquiring the representative fault features. Further, fault diagnosis of the diesel engine faces great challenges. This paper presents a new fault diagnosis method for the detection of diesel engine faults under multiple operation conditions instead of conventional methods confined to a single condition. First, an adaptive correlation threshold process is designed as a preprocessing unit to enhance data quality by weakening non-impact region characteristics. Next, a feature extraction method for sound signals based on the Mel frequency cepstrum (MFC) is improved and introduced into the machinery fault diagnosis. Then, the combination of the improved feature and vibrational mode decomposition (VMD) is proposed to incorporate VMD into an effective adaptive decomposition of non-stationary signals to combine it with an excellent feature representation of the vibration signal. Finally, the vector quantization algorithm is adopted to reduce the feature dimensions and generate codebook model bases, which trains the K-Nearest Neighbor classifiers. Five comparative methods were carried out, and the experimental results show that the proposed method offers a good effect of the common valve clearance fault of diesel engines under different conditions.

Smoke vehicle detection based on robust codebook model and robust volume local binary count patterns

Smoke vehicle detection is still vulnerable to false alarms. To reduce false alarm rates, this paper presents a smoke vehicle detection method based on Robust Codebook (R-Codebook) model and Robust Volume Local Binary Count (R-VLBC) pattern. In detecting suspected smoke blocks, we propose the R-Codebook model, which is robust to illumination changes and camera shaking or leaf shaking in the scene. In recognizing smoke block sequences, firstly, we propose a robust and discriminative descriptor called Non-Redundant VLBC (NR-VLBC). The feature dimension is reduced by half than the VLBC to overcome the issue that the VLBC is sensitive to the relative changes between background and foreground. Secondly, we propose a new Completed VLBC (CVLBC) by combining VLBC-Sign, VLBC-Magnitude, VLBC-Center gray level and VLBC-Difference to characterize spatial–temporal features. Thirdly, two strategies are used to further improve the proposed CVLBC to form R-VLBC. One is to overcome the issue that the CVLBC is sensitive to noise based on using a weighted local threshold (WLT), which is robust to noise and illumination variants and also make a balance between noise resistance and information of individual pixel. The other one is to extract multi-scale information by using a set of radius and sampling points. Extensive experiments show that the proposed method can achieve better performances than existing methods.

Extended Codebook with Multispectral Sequences for Background Subtraction.

The Codebook model is one of the popular real-time models for background subtraction. In this paper, we first extend it from traditional Red-Green-Blue (RGB) color model to multispectral sequences. A self-adaptive mechanism is then designed based on the statistical information extracted from the data themselves, with which the performance has been improved, in addition to saving time and effort to search for the appropriate parameters. Furthermore, the Spectral Information Divergence is introduced to evaluate the spectral distance between the current and reference vectors, together with the Brightness and Spectral Distortion. Experiments on five multispectral sequences with different challenges have shown that the multispectral self-adaptive Codebook model is more capable of detecting moving objects than the corresponding RGB sequences. The proposed research framework opens a door for future works for applying multispectral sequences in moving object detection.

Foreground detection using texture-based codebook method for monitoring systems

Monitoring systems have been extensively applied in daily life as a result of technological advancements. Foreground detection is an important technique to refine monitoring systems. Dynamic backgrounds in the environment form a rather challenging issue in foreground detection. For example, fluttering leaves, wavelets on water, flashes on monitor screens, shadow changes are all likely to lead to erroneous analysis in foreground detection. This study proposes a foreground detection algorithm, called texture-based codebook method, which using a codebook model in combination with texture features. The codebook model used to construct a background model can effectively compress data to achieve optimal processing speeds, whereas the local binary pattern is adopted to identify texture features because of its powerful capacity for texture description and fast computation. Subsequently, texture-based connected component labeling is applied to the results from the codebook model to increase accuracy and reduce error rates. In addition, to adapt to the environment, a short-term information model has been added for better background model updates. Gradient-based time difference is also incorporated to cope with global light changes. The experiment outcome shows that the algorithm proposed is capable of real-time processing and also has good adaptability. It is able to achieve decent recognition rates when tested in different environments.

Performance Analysis for Lossy-Forward Relaying Over Nakagami- $m$ Fading Channels

We investigate the performance of three-node lossy-forward (LF) relaying over independent block Nakagami- $m$ fading channels. Based on the theorem of source coding with side information, the exact outage probability expression for arbitrary values of the shape factor $m$ is derived under the assumptions of both the Gaussian codebook capacity and the constellation constrained capacity. The difference in outage probability between the two codebook models of capacity is found to be very minor. Furthermore, an accurate high signal-to-noise ratio approximation for the outage probability is obtained. It clearly identifies the equivalent diversity order and coding gain of the LF relaying. It is shown that the LF relaying is superior to conventional decode-and-forward relaying in terms of the outage probability and the $\epsilon$ -outage achievable rate. Moreover, with the LF relaying, the optimal location for the relay (R), which minimizes the outage probability, is found to be the point having same distance to the source (S) and the destination (D) when the S-R and R-D links experience the same level of fading. The accuracy of the analytical results is verified by a series of Monte Carlo simulations.

Sparse Codebook Model of Local Structures for Retrieval of Focal Liver Lesions Using Multiphase Medical Images.

Characterization and individual trait analysis of the focal liver lesions (FLL) is a challenging task in medical image processing and clinical site. The character analysis of a unconfirmed FLL case would be expected to benefit greatly from the accumulated FLL cases with experts' analysis, which can be achieved by content-based medical image retrieval (CBMIR). CBMIR mainly includes discriminated feature extraction and similarity calculation procedures. Bag-of-Visual-Words (BoVW) (codebook-based model) has been proven to be effective for different classification and retrieval tasks. This study investigates an improved codebook model for the fined-grained medical image representation with the following three advantages: (1) instead of SIFT, we exploit the local patch (structure) as the local descriptor, which can retain all detailed information and is more suitable for the fine-grained medical image applications; (2) in order to more accurately approximate any local descriptor in coding procedure, the sparse coding method, instead of K-means algorithm, is employed for codebook learning and coded vector calculation; (3) we evaluate retrieval performance of focal liver lesions (FLL) using multiphase computed tomography (CT) scans, in which the proposed codebook model is separately learned for each phase. The effectiveness of the proposed method is confirmed by our experiments on FLL retrieval.

배경 모델 갱신을 통한 코드북 기반의 전배경 분할

최근 코드북 기반의 전 배경 분리 알고리즘에 대한 연구가 활발히 진행되고 있다. 코드북은 입력 영상 시퀀스로부터 화소당 하나씩 만들어 지는데, 코드북 내의 각 코드워드는 동일 위치의 훈련 화소들을 대상으로 양자화를 수행한 클러스터 대표 벡터이다. 일반적인 코드북 기반 방법들은 초기 배경 모델 생성을 위하여 긴 시간동안 훈련 샘플들의 학습 과정을 거친다. 본 논문에서는 초기 몇 장의 프레임으로 부터 간단한 중위수 연산을 통하여 초기 배경 모델을 생성하고, 시간의 흐름에 따라 변화된 배경 정보를 포함할 수 있도록 코드워드의 사용 빈도수에 기반하여 배경 모델을 갱신한다. 제안한 알고리즘을 OpenCV 3.0과 연동하여 C언어로 구현하여 몇 개의 PETS2009 데이터에 적용 실험하였다. 해당 데이터는 준-주기적 움직임을 갖는 영상 시퀀스, 이동 물체의 일시 정지 등의 시나리오를 포함하고 있다. 실험을 통하여 제안한 방식이 GMM 알고리즘, 표준 코드북 알고리즘에 비하여 우수한 성능을 나타내고 있음을 확인하였다. Recently, a foreground-background segmentation using codebook model has been researched actively. The codebook is created one for each pixel in the image. The codewords are vector-quantized representative values of same positional training samples from the input image sequences. The training is necessary for a long time in the most of codebook-based algorithms. In this paper, the initial codebook model is generated simply using median operation with several image frames. The initial codebook is updated to adapt the dynamic changes of backgrounds based on the frequencies of codewords that matched to input pixel during the detection process. We implemented the proposed algorithm in the environment of visual c++ with opencv 3.0, and tested to some of the public video sequences from PETS2009. The test sequences contain the various scenarios including quasi-periodic motion images, loitering objects in the local area for a short time, etc. The experimental results show that the proposed algorithm has good performance compared to the GMM algorithm and standard codebook algorithm.

Adaptive maintenance scheme for codebook-based dynamic background subtraction

We propose a novel adaptive maintenance scheme for the codebook-based background subtraction algorithm. With this technique, the accuracy and efficiency of the model are significantly improved. In the proposed method, we develop an equal-qualification updating strategy to replace the maximum-negative-run-length-based filtering strategy. Further, we substitute the cache-based foreground learning process with a random updating scheme. These modifications not only preserve the accuracy of the codebook model but also significantly reduce the number of parameters used in the maintenance scheme. In the modified framework, parameters that are scenario-sensitive are identified through extensive experiments and analysis. Then, adaptive methods are proposed for them. The proposed method ensures the best performance of the system across a variety of complex scenarios. In our experiments, comparisons are provided to confirm that the performance of the codebook model is significantly improved owing to the adaptive technique. The overall performance of the proposed method is evaluated against more than 20 state-of-the-art methods using several modern datasets. It is demonstrated that, despite using only color information, the proposed method outperforms the majority of the solutions by a significant margin.

A novel infrared moving target detection algorithm based on multiscale codebook model

Traditional target detection algorithm based on codebook model only use pixel information of video image while spatial scale information of image is ignored, so the detection result usually has high false detection rate and the target’s characteristics is not obvious. To overcome this difficulty, a novel infrared (IR) moving target detection algorithm based on multiscale codebook model is presented in this paper. The main principle of this algorithm is to make full use of image pixel information and scale information for moving target detection. First, by Gauss pyramid image hierarchical model, the IR video is stratified into three layers, namely the original layer, the second layer and the top layer. Second, background codebook model is built for each layer image, the main feature information is discovered to update background codebook models, and then moving target in IR video is detected according to the updated background model. Finally, the fusion operation is done on detection results of three layers to get the final detection result. Compared with traditional detection algorithm based on codebook model, this IR target detection algorithm combines image pixel information and scale characteristics. By using this novel algorithm, the experiments on some real world IR images are performed. The whole algorithm implementing processes and results are analyzed, and this novel detection algorithm is evaluated from the two aspects: subjective evaluation and objective evaluation. From the experiment results, we can see that the proposed method has better detection effects, richer target information and lower false detection rate.

Fast Background Subtraction for Microorganism Detection in Wireless Visual Sensor Networks

With the emergency and development of portable microscopes, applying wireless network and image analysis techniques to real-time and intelligent surveillance in sewage treatment system has paid much attention recently. However, the resource limitation and processing capacity are the main constraints in wireless visual sensor networks (WVSNs), which make many background subtraction algorithms for moving object detection unable to work. In this paper, a more efficient and effective detection algorithm based on background subtraction has been proposed. Firstly, a new codebook model is initialized by selecting randomly in the neighbor of spatial domain from the first frame of video sequence, which greatly reduces the modeling time. Secondly, a new color space and a fast local directional pattern texture operator have been introduced into this model to increase the target contrast and neighborhood dependency. Finally, a self-adaptive model updating strategy based on statistical parameter estimation has been proposed for adding new code words and deleting noisy code words from models quickly and effectively. Experimental results have demonstrated that our method is able to achieve a better balance between effectiveness and efficiency compared with the state-of-the-art background subtraction algorithms.

Dynamic foreground detection based on improved Codebook model

Foreground detection method based on improved Codebook algorithm is discussed in this paper: first of all, transform RGB colour space into YCbCr colour space to make chromaticity convergent, and cope with illumination changing. Second, search for the CodeWords matching with the pixel values of the YCbCr to update the algorithm, so as to ensure the foreground detection effect. Then apply random abandon value method to delete the CodeWords that have not been accessed for a long time, reduce memory consumption and improve processing speed. Finally, conduct experiments on the infrared imagery and the colour image, respectively, to make foreground detection. The results demonstrate that the model and algorithm presented in this paper can make better foreground detection and reduce memory consumption. Meanwhile, compare this method with other algorithms to prove its advantages.

A Novel Object Detection Method Using Gaussian Mixture Codebook Model of RGB-D Information

A Novel Object Detection Method Using Gaussian Mixture Codebook Model of RGB-D Information

Double-Trapezium Cylinder Codebook Model Based on YUV Color Model for Foreground Detection with Shadow and Highlight Suppression

In order to solve the problem of existences of shadows and highlights for foreground detection in intelligent visual surveillance, Double-Trapezium Cylinder Codebook model based on YUV color model (DTCC_YUV) is proposed. First the shadow detection area is located in the lower part of the model based on the low luminance characteristic of shadows compared with backgrounds. An erected trapezium cylinder is constructed as the structure of the shadow detection area based on the variable chrominance characteristic of shadows. Second the highlight detection area is located in the upper part of the model based on the high luminance characteristic of highlights compared with backgrounds. An inverted trapezium cylinder is constructed as the structure of the highlight detection area based on the variable chrominance characteristic of highlights. Finally the main background area is constructed using a cylinder structure in the middle part of the model and then DTCC_YUV model is constructed completely. The experimental results show that DTCC_YUV model achieves high real-time performance with the fact that its average frame rate is about 22.53 frames per second which is higher than that of CBM, HC3, iGMM, GCBM, MCBS and HPB with 94.35 %, 146.71 %, 161.66 %, 145.56 %, 30.04 % and 17.88 % respectively. Also DTCC_YUV model achieves high shadow suppression performance with the fact that its average shadow suppression rate is about 59.35 % which is higher than that of CBM, HC3, iGMM, GCBM, MCBS and HPB with 6.59 %, 92.84 %, 413.16 %, 232.39 %, 27.35 % and 38.00 % respectively. At the same time DTCC_YUV model achieves high highlight suppression performance with the fact that its average highlight suppression rate is about 85.86 % which is higher than that of CBM, HC3, iGMM, GCBM, MCBS and HPB with 6.29 %, 17.92 %, 16.74 %, 152.00 %, 9.76 % and 12.59 % respectively.

A Self-adaptive CodeBook (SACB) model for real-time background subtraction

Effective and efficient background subtraction is important to a number of computer vision tasks. In this paper, we introduce a new background model that integrates several new techniques to address key challenges for background modeling for moving object detection in videos. The novel features of our proposed Self-adaptive CodeBook (SACB) background model are: a more effective color model using YCbCr color space, a statistical parameter estimation method, and a new algorithm for adding new background codewords into the permanent model and deleting noisy codewords from the models. Also, a new block-based approach is introduced to exploit the local spatial information. The proposed model is rigorously tested and has shown significant performance improvements over several previous models.

A novel visual codebook model based on fuzzy geometry for large-scale image classification

The codebook model has been developed as an effective means for image classification. However, the inherent operation of assigning visual words to image feature vectors in traditional codebook approaches causes serious ambiguities in image classification. In particular, the nearest word may not be the best fit to a feature, and multiple words may be equally appropriate for one specific feature. To resolve these ambiguities, we propose a novel visual codebook model based on the n-dimensional fuzzy geometry (n-D FG) theory, where all visual words and features are modeled as fuzzy points in the n-D FG space, and appropriate uncertainty is introduced to each fuzzy point to enhance the representation capacity. This n-D FG-codebook model not only inherits advantages from the fuzzy set theory, but also facilitates the analysis and determination of the relationship between visual words and features in geometric form. By explicitly taking into account the ambiguities, we propose a novel measure of similarity between the visual words and fuzzy features. Following the proposed codebook model and the novel similarity measure, we develop two useful image classification algorithms by modifying popular image coding algorithms (i.e. SPM and LLC). Finally, experimental results demonstrate that the classification accuracy of the proposed algorithms is dramatically improved for a standard large-scale image database. For example, with a codebook size of 256, the proposed algorithms achieve similar performance as traditional algorithms with a codebook size of 1024, indicating that the proposed algorithms reduce the computational cost by 75% while achieving almost identical classification accuracy to traditional algorithms. Thus, the proposed algorithms represent a more efficient and appropriate scheme for big image data.