Foreground Parts Research Articles

Furniture design based on image color extraction algorithm

With the increasing demand for personalized and customized home products, how to realize the innovative design of furniture and improve the design efficiency has become a research hotspot for related professionals. Aiming at these problems, the study extracts the main color of furniture images by optimizing the K-mean clustering algorithm, uses the simulated annealing algorithm to color-match the furniture, and reconstructs the image by edge detection to design a furniture design method based on image color extraction. The results revealed that in the foreground part, the correct rate of color match based on the design method was 95.7%, and in the background part, the correct rate of color match based on the design method was 94.81 %, which proved its effectiveness. The average feature point extraction time and the average feature point matching time of the design-based algorithm were 5.45 ms and 9.83 ms, respectively, which proved its high computational efficiency. In furniture color edge detection and overall color match, the image obtained based on the design method was significantly clearer, and the overall coherence, saturation and brightness were closer to the input image. In addition to raising the standard of furniture design, the study's design methodology increases design efficiency and offers solid technical support for the area.

Automatic non-destructive UAV-based structural health monitoring of steel container cranes

Container cranes are of key importance for maritime cargo transportation. The uninterrupted and all-day operation of these container cranes, which directly affects the efficiency of the port, necessitates the continuous inspection of these massive hoisting steel structures. Due to the large size of cranes, the current manual inspections performed by expert climbers are costly, risky, and time-consuming. This motivates further investigations on automated non-destructive approaches for the remote inspection of fatigue-prone parts of cranes. In this paper, we investigate the effectiveness of color space-based and deep learning-based approaches for separating the foreground crane parts from the whole image. Subsequently, three different ML-based algorithms (k-Nearest Neighbors, Random Forest, and Naive Bayes) are employed to detect the rust and repainting areas from detected foreground parts of the crane body. Qualitative and quantitative comparisons of the results of these approaches were conducted. While quantitative evaluation of pixel-based analysis reveals the superiority of the k-Nearest Neighbors algorithm in our experiments, the potential of Random Forest and Naive Bayes for region-based analysis of the defect is highlighted.

An Efficient Approach of Assessing Quality of Blurred Image

This paper presents a method for evaluating the quality of images altered by Gaussian blur. The method is based on the observation of bokeh mode images where the region of interest (foreground) is sharp, while the remaining parts (background) are intentionally blurred to enhance the perceptual quality of the image. The blurriness of the background increases attention towards the foreground part of the image. The proposed quality metric is obtained by combining the attention factor and the sharpness of the region of interest. The accuracy, in terms of Spearman’s-rank-order correlation-coefficient (SROCC), for popular and publicly available databases such as LIVE, VCL, TID2008, CSIQ, and TID2013, is 0.963, 0.925, 0.900, 0.930, and 0.930, respectively. The proposed method achieves high and consistent Spearman’s rank-order correlation coefficient (SROCC) values compared to the majority of state-of-the-art algorithms. Furthermore, in terms of speed, the proposed method surpasses other state-of-the-art methods. The MATLAB code of the proposed metric is publicly available at https://drive.google.com/drive/folders/1SRmUp0N157Ati9l3kV13uoCxw5PhMgQn?usp=sharing .

Infrared image enhancement based on adaptive non-local filter and local contrast

High-quality infrared images urgently needed in military and civilian fields are always associated with appropriate contrast. However, the main challenge to obtain high-quality infrared images lies in enhancement of the contrast effectively without over-enhancement of the background and noise. Hereby, an effective infrared image enhancement approach is proposed and based on an adaptive non-local filter and local contrast. An input infrared image is detected by noisy adaptive detection to separate noisy pixels, which are filtered by non-local means filter to acquire denoised image. The grayscale histogram of the denoised image is divided into foreground and background based on the local minima value. The foreground part is enhanced by local contrast weighted distribution and max entropy gamma correction. The background part is processed by linear mapping to project the grayscale to an appropriate region. Finally, the enhanced infrared image is obtained by remapping the processed foreground and background histogram. Extensive experiments on public and homemade datasets demonstrate that our method achieves 46.8183 on image clarity, which expresses its superiority for infrared image enhancement.

Oriented Object Detection Based on Foreground Feature Enhancement in Remote Sensing Images

Oriented object detection is a fundamental and challenging task in remote sensing image analysis and has received much attention in recent years. Optical remote sensing images often have more complex background information than natural images, and the number of annotated samples varies in different categories. To enhance the difference between foreground and background, current one-stage object detection algorithms attempt to exploit focus loss to balance the foreground and background weights, thus making the network more focused on the foreground part. However, the current one-stage object detectors still face two main challenges: (1) the detection network pays little attention to the foreground and does not make full use of the foreground information; (2) the distinction of similar object categories has not attracted attention. To address the above challenges, this paper presents a foreground feature enhancement method applied to one-stage object detection. The proposed method mainly includes two important components: keypoint attention module (KAM) and prototype contrastive learning module (PCLM). The KAM is used to enhance the features of the foreground part of the image and reduce the features of the background part of the image, and the PCLM is utilized to enhance the discrimination of samples between foreground categories and reduce the confusion of samples between different categories. Furthermore, the proposed method designs and adopts an equalized modulation focal loss (EMFL) to optimize the training process of the model and increase the loss weight of the foreground later in the model training. Experimental results on the publicly available DOTA datasets and HRSC2016 datasets show that our method exhibits state-of-the-art performance.

Quaternion-based dynamic mode decomposition for background modeling in color videos

Scene Background Initialization (SBI) is one of the challenging problems in computer vision. Dynamic mode decomposition (DMD) is a recently proposed method to robustly decompose a video sequence into the background model and the corresponding foreground part. However, this method needs to convert the color image into a grayscale image for processing, which leads to the neglect of the coupling information between the three channels of the color image. In this study, we propose a quaternion-based DMD (Q-DMD), which extends the DMD by quaternion matrix analysis, so as to ultimately preserve the inherent color structure of the color image and the color video. We exploit the standard eigenvalues of the quaternion matrix to compute its spectral decomposition and calculate the corresponding Q-DMD modes and eigenvalues. The results on the publicly available benchmark datasets prove that our Q-DMD outperforms the exact DMD method, and experiment results also demonstrate that the performance of our approach is comparable to that of the state-of-the-art ones.

FEXNet: Foreground Extraction Network for Human Action Recognition

As most human actions in video sequences embody the continuous interactions between foregrounds rather than the background scene, it is significant to disentangle these foregrounds from the background for advanced action recognition systems. In this paper, therefore, we propose a Foreground EXtraction (FEX) block to explicitly model the foreground clues to achieve effective management of action subjects. In particular, the designed FEX block contains two components. The first part is a Foreground Enhancement (FE) module, which highlights the potential feature channels related to the action attributes, providing channel-level refinement for the following spatiotemporal modeling. The second phase is a Scene Segregation (SS) module, which splits feature maps into foreground and background. Specifically, a temporal model with dynamic enhancement is constructed for the foreground part, reflecting the essential nature of the action category. While the background is modeled using simple spatial convolutions, mapping the inputs to the consistent feature space. The FEX blocks can be inserted into existing 2D CNNs (denoted as FEXNet) for spatiotemporal modeling, concentrating on the foreground clues for effective action inference. Our experiments performed on Something-Something V1, V2 and Kinetics400 verify the effectiveness of the proposed method.

Three-Dimension Measurement of Mechanical Parts Based on Structure from Motion(SfM) Algorithm

Background: As an important branch of computer vision, visual measurement is a fast developing cutting-edge technology, which has been widely used in the manufacturing field. In recent years, the visual measurement of feature size of probes through small IC probes has aroused wide concern. Objective: This study aims to take small shaft parts as the research object in order to provide a full set of novel and reliable technical means for the three-dimension measurement of mechanical parts. Methods: Firstly, the trinocular vision measurement system based on the curved cantilever mechanism was designed and constructed. Secondly, the measurement system was used to collect the part images from different angles, and the images derived from the four categories of segmentation algorithms such as threshold-based, region-based segmentation algorithm were compared and analyzed. Lazy Snapping image segmentation algorithm was used to extract the foreground parts of each image. After comparing and analyzing SfM-based algorithm and Visual Hull-based algorithm, the SfM-based algorithm was adopted to reconstruct the 3D morphology of the parts. The measurement of the relevant dimensions was performed. Results: The results shows that Lazy Snapping's human-computer interaction brush function improves the accuracy and stability of image segmentation of different algorithms, such as threshold value method, regional method, Grab Cut, and Dense Cut. The SfM-based 3D reconstruction algorithm is of high robustness and fast speed. Conclusion: This study provides an effective method for measuring small mechanical parts, which will shorten the measurement cycle, improve the measurement speed, and reduce the measurement cost.

A machine learning‐based method for automatic diagnosis of ankle fracture using X‐ray images

AbstractDue to the complex mechanism of ankle injury, the clinical diagnosis of ankle fracture is extremely difficult. In order to simplify the fracture diagnosis process, this study proposes an automatic diagnosis model of ankle fractures. Firstly, an ankle fracture classification method suitable for machine learning was developed. By dividing six fracture regions, multiple types of fractures were clarified, and a corresponding dataset was created accordingly. Secondly, the random forest model was used to preprocess the X‐ray images and segment the fracture foreground part of the X‐ray images. Finally, the bag of visual words (BoVW) was used for feature extraction, and classifiers were constructed in different regions for classification. The area under the curve (AUC) of XGBoost was 0.92 ± 0.06. The performance of the XGBoost has proved to be better compared with the SVM when training on a small dataset in each region.

Saliency prediction on omnidirectional images with attention-aware feature fusion network

Recent years have witnessed rapid development of deep learning technology and its successful application in the saliency prediction of traditional 2D images. However, when using deep neural network (DNN) models to perform saliency prediction on omnidirectional images (ODIs), there are two critical issues: (1) The datasets for ODIs are small-scale that cannot support the training DNN-based models. (2) It is challenging to perform saliency prediction in that some ODIs contain complex background clutters. In order to solve these two problems, we propose a novel Attention-Aware Features Fusion Network (AAFFN) model which is first trained with traditional 2D images and then transferred to the ODIs for saliency prediction. Specifically, our proposed AAFFN model consists of three modules: a Part-guided Attention (PA) module, a Visibility Score (VS) module, and a Attention-Aware Features Fusion (AAFF) module. The PA module is used to extract precise features to estimate attention of the finer part on ODIs, and eliminate the influence of cluttered background. Meanwhile, the VS module is introduced to measure the proportion of the foreground and background parts and generate visibility scores in the feature learning process. Finally, in the AAFF module, we utilize the weighted fusion of attention maps and visibility scores to generate the final saliency map. Extensive experiments and ablation analysis demonstrate that the proposed model achieves superior performance and outperforms other state-of-the-art methods on public benchmark datasets.

To See in the Dark: N2DGAN for Background Modeling in Nighttime Scene

Due to the deteriorated conditions of illumination lack and uneven lighting, the performance of traditional background modeling methods is greatly limited for the surveillance of nighttime video. To make background modeling under nighttime scene performs as well as in daytime condition, we put forward a promising generation-based background modeling framework for foreground surveillance. With a pre-specified daytime reference image as background frame, the GAN based generation model, called N2DGAN, is trained to transfer each frame of nighttime video to a virtual daytime image with the same scene to the reference image except for the foreground part. Specifically, to balance the preservation of background scene and the foreground object(s) in generating the virtual daytime image, we presented a two-pathway generation model, in which the global and local sub-networks were well combined with spatial and temporal consistency constraints. For the sequence of generated virtual daytime images, a multi-scale Bayes model was further proposed to characterize pertinently the temporal variation of background. We manually labeled ground truth on the collected nightime video datasets for performance evaluation. The impressive results illustrated in both the main paper and supplementary show the effectiveness of our proposed approach.

A Tensor Foreground-Background Separation Algorithm Based on Dynamic Dictionary Update and Active Contour Detection

Foreground-background separation of surveillance video, that models static background and extracts moving foreground simultaneously, attracts increasing attentions in building a smart city. Conventional techniques towards this always consider the background as primary target and tend to adopt low-rank constraint as its estimator, which provides finite (equal to the value of rank) alternatives when constructing the background. However, in practical missions, although general sketch of background is stable, some details change constantly. Aimed at this, we propose to represent the general background by a linear combination of some atoms and record the detailed background by spatiotemporal clustered patches. Then, the moving foreground is considered as a mixture of active contours and continuous contents. Eventually, joint optimization is conducted under a unified framework, i.e., alternating direction multipliers method (ADMM), and produces our tensor model for hierarchical background and hierarchical foreground separation (THHS). The employed tensor space, which agrees with the instinct structure of video data, benefits all the spatiotemporal designs in both background modular and foreground part. Experimental results show that THHS is more adaptive to the dynamic background and produces more accurate foreground when compared against current state-of-the-art techniques.

The Proscenium of Opera Houses as a Disappeared Intangible Heritage: A Virtual Reconstruction of the 1840s Original Design of the Alighieri Theatre in Ravenna

In a Historical Opera House (HOH), the proscenium is the foreground part of the stage. Until the end of the 19th Century, it was extended through the cavea, being the orchestra placed at the same level of the stalls, without an orchestra pit. Soloists often moved in the proscenium when they sung, in order to increase the strength of the voice and the intelligibility of the text. The Alighieri theatre in Ravenna, designed by the Meduna brothers, the former designers of Venice’s “La Fenice” theater, is chosen as a case study. During a refurbishment in 1928, the proscenium of the stage was removed in order to open the orchestra pit, which was not considered in the original design. The original design and the present one are compared by using numerical simulations. Acoustic measurements of the opera house and vibro-acoustic measurements on a wooden stage help to reach a proper calibration of both models. Results are discussed by means of ISO 3382 criteria: the proscenium increases the sound strength of the soloists but reduces the intelligibility of the text.

Robust Foreground Estimation via Structured Gaussian Scale Mixture Modeling.

Recovering the background and foreground parts from video frames has important applications in video surveillance. Under the assumption that the background parts are stationary and the foreground are sparse, most of existing methods are based on the framework of robust principal component analysis (RPCA), i.e., modeling the background and foreground parts as a low-rank and sparse matrices, respectively. However, in realistic complex scenarios, the conventional norm sparse regularizer often fails to well characterize the varying sparsity of the foreground components. How to select the sparsity regularizer parameters adaptively according to the local statistics is critical to the success of the RPCA framework for background subtraction task. In this paper, we propose to model the sparse component with a Gaussian scale mixture (GSM) model. Compared with the conventional norm, the GSM-based sparse model has the advantages of jointly estimating the variances of the sparse coefficients (and hence the regularization parameters) and the unknown sparse coefficients, leading to significant estimation accuracy improvements. Moreover, considering that the foreground parts are highly structured, a structured extension of the GSM model is further developed. Specifically, the input frame is divided into many homogeneous regions using superpixel segmentation. By characterizing the set of sparse coefficients in each homogeneous region with the same GSM prior, the local dependencies among the sparse coefficients can be effectively exploited, leading to further improvements for background subtraction. Experimental results on several challenging scenarios show that the proposed method performs much better than most of existing background subtraction methods in terms of both performance and speed.

Automatic Solitary Lung Nodule Detection in Computed Tomography Images Slices

Lung nodule is an early indicator of some lung diseases, including lung cancer. In Computed Tomography (CT) based image, nodule is known as a shape that appears brighter than lung surrounding. This research aim to develop an application that automatically detect lung nodule in CT images. There are some steps in algorithm such as image acquisition and conversion, image binarization, lung segmentation, blob detection, and classification. Data acquisition is a step to taking image slice by slice from the original *.dicom format and then each image slices is converted into *.tif image format. Binarization that tailoring Otsu algorithm, than separated the background and foreground part of each image slices. After removing the background part, the next step is to segment part of the lung only so the nodule can localized easier. Once again Otsu algorithm is use to detect nodule blob in localized lung area. The final step is tailoring Support Vector Machine (SVM) to classify the nodule. The application has succeed detecting near round nodule with a certain threshold of size. Those detecting result shows drawback in part of thresholding size and shape of nodule that need to enhance in the next part of the research. The algorithm also cannot detect nodule that attached to wall and Lung Chanel, since it depend the searching only on colour differences.

Adapting Convolutional Neural Network to Multi-label Image Classification

Previous Multi-label image classification is largely limited by the representation power of the hand-crafted features. The convolutional neural network (CNN) has achieved successes in many computer vision tasks. In this work, we adapt the CNN to the multi-label image classification, where three approaches are used including end-to-end training on the target dataset, pre-training on Image Net and fine-tuning on the target dataset, CNN features extracted from Image Net for the AdaBoost.MH classifier. The experimental results on two datasets show that CNN model can boost large margin on the object dataset in contrast with the hand-crafted features methods, which achieves at most 98% on MSRC compared to 92% by the state-of-art algorithms, but benefit a little in the scene dataset. The primary discrepancies between the object and the scene classification tasks lies in that the former need to only focus on the foreground part of the image but the latter requires paying attention to the entire image. Source code upon Torch7 toolkit to reproduce the experiments in the paper is made publicly available.

Research on Background Modeling Based on Low-Rank Matrix Recovery

Key and difficult points of background subtraction method lie in looking for an ideal background modeling under complex scene. Stacking the individual frames as columns of a big matrix, background parts can be viewed as a low-rank background matrix because of large similarity among individual frames, yet foreground parts can be viewed as a sparse matrix as foreground parts play a small role in individual frames. Thus the process of video background modeling is in fact a process of low-rank matrix recovery. Background modeling based on low-rank matrix recovery can separate foreground images from background at the same time without pre-training samples, besides, the approach is robust to illumination changes. However, there exist some shortcomings in background modeling based on low-rank matrix recovery by analyzing numerical experiments, which is developed from three aspects.

A New Framework for Direct Saliency Detection and Segmentation Based on Graph Methods

Saliency detection is an important research topic in computer vision. The traditional methods compute image saliency map, then salient segmentation is based on the corresponding saliency map. Unfortunately, overall performance of this method is poor due to the reason of losing some fine details and spatial information within image. This paper presents a new framework to overcome the drawback, named FDSRDS(Framework for Directly Salient Region Detection and Segmentation based on graph methods). Under FDSRSD, firstly, we get the foreground image by segmenting the original image via our extended grabcut algorithm. Mostly, the saliency region is within the foreground part. Secondly, we segment the foreground image into regions by means of graph based segmentation and nearest neighbor graph . Thirdly, we use relative weber's luminance rules to calculate every region’s luminance. Finally, we get the maximum luminance region which is the saliency region. Under FDSRSD framework, algorithms we proposed capture fine details and spatial relationships in saliency computation. We demonstrate impressive results by evaluating our method with other five state-of-the-art methods on the publicly available data set.

Region based Coding of 3D Magnetic Resonance Images for Telemedicine Applications

ABSTRACT Region Based Coding (RBC) technique is significant for medical image compression and transmission. Lossless compression schemes with secure transmission play a key role in telemedicine applications that help in accurate diagnosis and research. In this paper we propose a lossless compression approach based on 3D integer wavelet transform, 3D SPIHT algorithm of MR images. The use of lifting scheme allows to generate truly lossless integer to integer wavelet transforms. The main objective of this work is rejects the noisy background and reconstructs the image portion losslessly. In this work different integer wavelet transforms will be used to compress the 3D MR images. The performance of the system has been evaluated based on bits-per-pixel and peak signal-to-noise ratio. Keywords 3D SPIHT algorithm, Integer Wavelet Transform, Lossless compression, Medical Image Compression, Region Based Coding. 1. INTRODUCTION Image compression is useful reducing the storage and transmission bandwidth requirements of medical imges.Compression methods are classified into lossless and lossy methods. In the medical image scenario, lossy compression schemes are not generally used. This is due to possible loss of useful clinical information which may influence diagnosis. In addition to these reasons, there can be legal issues. Storage of medical images is generally problematic because of the requirement to preserve the best possible image quality which is usually interpreted as a need for lossless compression. 3D magnetic resonance image (MRI) data, which contains multiple slices representing a part of a body, requires compression for efficient storage and transmission. Compression of medical data is also required in telemedicine applications where image data needs to be transmitted over the network. Lossless compression, Progressive transmission and region of interest (ROI) are important functionalities for a compression scheme. Wavelet-based techniques are the latest development in the field of image compression. It offers multiresolution capability that is not available in any of the other methods. The wavelet transform analyzes a signal in time and scale. The low frequency components in the signal that are spread out in time, as well as the high frequency components that are localized in time are captured by a variable-length window [3]. The window is shifted by different units of time in a discrete manner, thus covering the entire signal. Lifting is a technique used in constructing second generation wavelets, entirely in the spatial domain [6]. The first generation wavelets are translates and dilates of a single mother wavelet, and Fourier techniques are essential in their construction. The lifting scheme does not use the Fourier technique [5]. It is a very fast method compared to the first generation wavelets. Moreover, the inverse transform is obtained by replacing addition with subtraction and reversing the operations in the forward transform. The goal of quantization is to encode the data from a source, with some loss, so that the best reproduction is obtained. Set partitioning hierarchical trees (SPIHT) achieves more compression than EZW [4].A typical MR image foreground contains clinical information which needs to be compressed without any loss. On the other hand the background does not contain any clinical information. It is only noise and consumes unnecessary bit budget and impairs the performance of a compression scheme. In this work we ignore the background part. We generate image masks in such a way that the foreground part is totally included and the pixel values in the foreground part are made zero. Morphological operations can be effectively used to generate image masks, which contain a value of „1‟ in the foreground and a value of „0‟ in the background. The original image is then multiplied with these masks to obtain “background noise free” images while keeping the information in the foreground part. In this paper we compare proposed work using different integer wavelets with the original scheme using 3DSPIHT algorithm.

Multi-oriented Bangla and Devnagari text recognition

There are printed complex documents where text lines of a single page may have different orientations or the text lines may be curved in shape. As a result, it is difficult to detect the skew of such documents and hence character segmentation and recognition of such documents are a complex task. In this paper, using background and foreground information we propose a novel scheme towards the recognition of Indian complex documents of Bangla and Devnagari script. In Bangla and Devnagari documents usually characters in a word touch and they form cavity regions. To take care of these cavity regions, background information of such documents is used. Convex hull and water reservoir principle have been applied for this purpose. Here, at first, the characters are segmented from the documents using the background information of the text. Next, individual characters are recognized using rotation invariant features obtained from the foreground part of the characters. For character segmentation, at first, writing mode of a touching component (word) is detected using water reservoir principle based features. Next, depending on writing mode and the reservoir base-region of the touching component, a set of candidate envelope points is then selected from the contour points of the component. Based on these candidate points, the touching component is finally segmented into individual characters. For recognition of multi-sized/multi-oriented characters the features are computed from different angular information obtained from the external and internal contour pixels of the characters. These angular information are computed in such a way that they do not depend on the size and rotation of the characters. Circular and convex hull rings have been used to divide a character into smaller zones to get zone-wise features for higher recognition results. We combine circular and convex hull features to improve the results and these features are fed to support vector machines (SVM) for recognition. From our experiment we obtained recognition results of 99.18% (98.86%) accuracy when tested on 7515 (7874) Devnagari (Bangla) characters.