Single Image Frame Research Articles

Robust Principal Component Analysis-Based Infrared Small Target Detection

A method based on Robust Principle Component Analysis (RPCA) technique is proposed to detect small targets in infrared images. Using the low rank characteristic of background and the sparse characteristic of target, the observed image is regarded as the sum of a low-rank background matrix and a sparse outlier matrix, and then the decomposition is solved by the RPCA. The infrared small target is extracted from the single-frame image or multi-frame sequence. In order to get more efficient algorithm, the iteration process in the augmented Lagrange multiplier method is improved. The simulation results show that the method can detect out the small target precisely and efficiently.

Infrared dim and small target detection based on two-stage U-skip context aggregation network with a missed-detection-and-false-alarm combination loss

Infrared small target detection (ISTD) is a critical technique in both civil and military applications such as leak and defect inspection, cell segmentation for medicine analysis, early-warning systems and so on. Over the last decade, numerous ISTD methods have been proposed, such as methods based on image denoising, visual saliency detection, low-rank matrix recovery and traditional machine learning, but training an end-to-end deep model to detect small targets has not been fully investigated. In this regard, the paper proposes a novel deep model called UCAN for ISTD which concatenates two context aggregation networks and connects them using U-skip connections. A Missed-detection-and-False-alarm Combination(MFC) loss function, which is based on the Neyman-Pearson decision theory, is proposed to train the model and can well balance the detection rate and the false alarm rate. In addition, a two-stage detection scheme which involves a cascade of two UCANs is proposed to further improve the overall detection performance of ISTD. Extensive experiments on real infrared sequences and a single-frame image set and the comparison with state-of-the-art methods demonstrate the superiority of the proposed model.

Facial Expression Recognition Based on a Hybrid Model Combining Deep and Shallow Features

Facial expression recognition plays an important role in the field involving human-computer interactions. Given the wide use of convolutional neural networks or other neural network models in automatic image classification systems, high-level features can be automatically learned by hierarchical neural networks. However, the training of CNNs requires large amounts of training data to permit adequate generalization. The traditional scale-invariant feature transform (SIFT) does not need large learning samples to obtain features. In this paper, we proposed a feature extraction method for use in the facial expressions recognition from a single image frame. The hybrid features use a combination of SIFT and deep learning features of different levels extracted from a CNN model. The combined features are adopted to classify expressions using support vector machines. The performance of proposed method is tested using the publicly available extended Cohn-Kanade (CK+) database. To evaluate the generalization ability of our method, several experiments are designed and carried out in a cross-database environment. Compared with the 76.57% accuracy obtained using SIFT-bag of features (BoF) features and the 92.87% accuracy obtained using CNN features, we achieve a FER accuracy of 94.82% using the proposed hybrid SIFT-CNN features. The results of additional cross-database experiments also demonstrate the considerable potential of combining shallow features with deep learning features, and these results are more promising than state-of-the-art models. Combining shallow and deep learning features is effective when the training data are not sufficient to obtain a deep model with considerable generalization ability.

Super-resolution Reconstruction Using Multiconnection Deep Residual Network Combined an Improved Loss Function for Single-frame Image

Super-resolution reconstruction plays an important role in reconstructing image detail and improving image visual effects, which is achieved mostly by linear interpolation algorithm in the existing display equipment, and there are no obvious details and blurred edges. In this paper, a multi-connection convolution network is proposed to solve the problems of the super-resolution reconstruction (SR) algorithm. The network constructs a multi-connection network structure, where a long skip strategy is used to obtain the identity mapping. It can concatenate the low-level features and high-level features, and can simultaneously represent various complex reconstruction scenes. In addition, a strong and flexible two parameter loss function is used to optimize and train the deep network and improve the generalization ability of the network model. The simulation results show that the proposed SR algorithm in this paper can generate high resolution images with rich details and clearness, and the objective quantitative evaluation is greatly improved.

Early smoke detection of forest fires based on SVM image segmentation

A smoke detection method is proposed in single-frame video sequence images for forest fire detection in large space and complex scenes. A new superpixel merging algorithm is further studied to improve the existing horizon detection algorithm. This method performs Simple Linear Iterative Clustering (SLIC) superpixel segmentation on the image, and the over-segmentation problem is solved with a new superpixel merging algorithm. The improved sky horizon line segmentation algorithm is used to eliminate the interference of clouds in the sky for smoke detection. According to the spectral features, the superpixel blocks are classified by support vector machine (SVM). The experimental results show that the superpixel merging algorithm is efficient and simple, and easy to program. The smoke detection technology based on image segmentation can eliminate the interference of noise such as clouds and fog on smoke detection. The accuracy of smoke detection is 77% in a forest scene, it can be used as an auxiliary means of monitoring forest fires. A new attempt is given for forest fire warning and automatic detection.

Motion estimation and quality enhancement for a single image in dynamic single-pixel imaging.

In single-pixel imaging (SPI), a large number of illuminations is usually required to capture one single image. Consequently, SPI may only achieve a very low frame rate for a fast-moving object and the reconstructed images are contaminated with blur and noise. In previous works, some attempts are made to perform motion estimation between neighboring frames in a SPI video to enhance the image quality. However, the motion estimation and quality enhancement from one single image frame in dynamic SPI was seldom investigated. In this work, it assumed that some prior knowledge about the type of motion the object undergoes is known. A motion model of the target object is constructed and the motion parameters can be optimized within a search space. Our proposed scheme is different from common motion deblur techniques for photographs since the motion blur mechanism in SPI is significantly different from a conventional camera. Experimental results demonstrate that the reconstructed images with our proposed scheme in dynamic SPI have much better quality.

Study on Pointing Accuracy Effect on Image Quality of Space-borne Video SAR

Effect of satellite pointing accuracy on spaceborne video SAR imaging quality has been studied in this paper to solve the problem of indicators decomposition. First, the effect model of pointing error on the single frame image quality was established based on the rough and precise imaging. Then the relationship between the stability of Video SAR and pointing accuracy error has been analysised. Finally, the simulations prove the validity of the analysis. The analysis and the simulation results show that the attitude pointing error mainly affects the resolution, sidelobe ratio, ambiguity and radiation performance. For the video mode, the shift and rotation of the beam covering area were caused by the attitude pointing error. The result of the report can be used to distribute the system error and establish the index system.

Dim small target detection based on convolutinal neural network in star image

The detection of dim target in star image is a challenging task because of the low SNR target and complex background. In this paper, we present a deep learning approach to detecting dim small targets in single-frame star image under uneven background and different kinds of noises. We propose a fully convolutional neural network to achieve pixel-wise classification, which can complete target-background separation in a single stage rapidly. To train this network, we also build a synthetic star image dataset covering various noises and background distribution. The precise annotations of the target regions and centroid positions provided by this dataset make the supervised learning approach possible. Experimental results show that the proposed method outperforms the state-of-the-art in terms of higher detection rate and less false alarm caused by noises.

Coronary Calcium Detection Based on Improved Deep Residual Network in Mimics.

Coronary calcium detection in medicine image processing is a hot research topic. According to the low resolution and complex background in medicine image, an improved coronary calcium detection algorithm based on the Single Shot MultiBox Detector (SSD) in Mimics is proposed in this paper. The algorithm firstly uses the aggregate channel feature model to preprocess the image to obtain the suspected calcium area, which greatly reduces the time of single-frame image detection. The basic network VGG-16 is replaced by Resnet-50, which introduces the identity mapping to solve the problem of reducing the detection accuracy when the number of network layers are increased. Finally, the powerful and flexible two-parameter loss function is used to optimize the training deep network and improve the network model generalization ability. Qualitative and quantitative experiments show that the performance of the proposed detection algorithm exceeds the existing calcium detection algorithms, and the detection efficiency is improved while ensuring the accuracy of calcium detection.

Infrared small target detection based on an image-patch tensor model

Infrared small target detection under complex background has been applied to many fields and is still a challenging problem. In this paper, a small target detection method base on an image-patch tensor (IPT) model is proposed. Firstly, considering the structural relationship between pixels, the original single-frame image is constructed as a new image-patch tensor. Secondly, based on the correlation of the background image patch and the sparsity of the target image patch, the small target detection problem can be transformed into an optimization problem of separating the low-rank part and the sparse part of the tensor. Finally, after simple post filtering, the target is separated adaptively. Experimental results show that the proposed method can detect the small target precisely and can keep a higher signal-to-clutter ratio (SCR). What’s more, compared with other methods, the proposed method shows better detection performance.

Real-time Volumetric Assessment of the Human Carotid Artery: Handheld Multispectral Optoacoustic Tomography

Background Multispectral optical imaging has the capability of resolving hemoglobin, lipid, and water. Volumetric multispectral optoacoustic tomography (MSOT) is a hybrid imaging technique that provides a unique combination of functional and molecular contrast with real-time handheld imaging. Purpose To investigate whether volumetric MSOT can provide real-time assessment of the anatomic and functional status of the human carotid artery bifurcation noninvasively. Materials and Methods Imaging of healthy volunteers (n = 16) was performed with a custom-designed handheld volumetric MSOT scanner capable of high-spatial-resolution (approximately 200 µm) and real-time (10 volumes/sec) three-dimensional imaging, while further providing spectroscopic capacity through fast tuning of the excitation light wavelength. For comparison and anatomic cross-validation, volunteers were also scanned with clinical B-mode US. Results Volumetric MSOT achieved real-time imaging and characterization of the entire carotid bifurcation area across three dimensions simultaneously captured in a single volumetric image frame. Analysis of the acquired data further showed that a higher contrast-to-noise ratio can be achieved for wavelengths corresponding to a high optical absorption of oxygenated hemoglobin. Conclusion The human carotid artery was visualized by using handheld volumetric multispectral optoacoustic tomography. This imaging approach is less prone to motion artifacts than are the conventional clinical imaging methods, holding promise for providing additional image-based biomarkers for noninvasive label-free assessment of carotid artery disease. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Mezrich in this issue.

Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM

In view of the high dimensionality, nonrigidity, multiscale variation and the influence of illumination and angle on facial expressions, it is quite difficult to obtain facial expression images or videos using computers and analyze facial morphology and changes to accurately obtain the emotional changes of the subjects. Existing facial expression recognition algorithms have the following problems in the application process: the existing shallow feature extraction model has lost a lot of effective feature information and low recognition accuracy. The facial expression recognition method based on deep learning has problems such as overfitting, gradient explosion and parameter initialization. Therefore, this paper develops a facial expression recognition algorithm based on the deep learning method. An adaptive model parameter initialization based on the multilayer maxout network linear activation function is proposed to initialize the convolutional neural network (CNN) and the long–short-term memory network (LSTM) method. It can effectively overcome the gradient disappearance and gradient explosion problems in the deep learning model training process. At the same time, the convolutional neural network with an LSTM memory unit is used to extract the related information from the image sequence, and the facial expression judgment is based on a single-frame image and historical-related information. However, the top-level structure of the CNN model is a fully connected feedforward neural network, which undertakes the task of expression classification. Therefore, the SVM classification method replaces the top-level classifier to further improve the expression classification accuracy. Experiments show that the facial expression recognition method proposed in this paper not only accurately identifies various expressions but also has good adaptive ability. This is because the method achieves the adaptive initialization of the parameters of the deep learning model construction process and also analyzes the relevance of the expression database expression, thereby improving the accuracy of expression recognition.

Unobtrusive Behavioral Analysis of Students in Classroom Environment Using Non-Verbal Cues

Pervasive intelligent learning environments can be made more personalized by adapting the teaching strategies according to the students’ emotional and behavioral engagements. The students’ engagement analysis helps to foster those emotions and behavioral patterns that are beneficial to learning, thus improving the effectiveness of the teaching-learning process. Unobtrusive student engagement analysis is performed using the students’ non-verbal cues such as facial expressions, hand gestures, and body postures. Though there exist several techniques for classifying the engagement of a single student present in a single image frame, there are limited works on the students’ engagement analysis in a classroom environment. In this paper, we propose a convolutional neural network architecture for unobtrusive students’ engagement analysis using non-verbal cues. The proposed architecture is trained and tested on faces, hand gestures and body postures in the wild of more than 350 students present in a classroom environment, with each test image containing multiple students in a single image frame. The data annotation is performed using the gold standard study, and the annotators reliably agree with Cohen’s <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\kappa = 0.43$ </tex-math></inline-formula> . We obtained 71% accuracy for the students’ engagement level classification. Further, a pre-test/post-test analysis was performed, and it was observed that there is a positive correlation between the students’ engagement and their test performance.

Complexity estimation of image sequence for automatic target track

In the field of automatic target recognition and tracking, traditional image complexity metrics, such as statistical variance and signal-to-noise ratio, all focus on single-frame images. However, there are few researches about the complexity of image sequence. To solve this problem, a criterion of evaluating image sequence complexity is proposed. Firstly, to characterize this criterion quantitatively, two metrics for measuring the complexity of image sequence, namely feature space similarity degree of global background (FSSDGB) and feature space occultation degree of local background (FSODLB) are developed. Here, FSSDGB reflects the ability of global background to introduce false alarms based on feature space, and FSODLB represents the difference between target and local background based on feature space. Secondly, the feature space is optimized by the grey relational method and relevant features are removed so that FSSDGB and FSODLB are more reasonable to establish complexity of single-frame images. Finally, the image sequence complexity is not a linear sum of the single-frame image complexity. Target tracking errors often occur in high-complexity images and the tracking effect of low-complexity images is very well. The nonlinear transformation based on median (NTM) is proposed to construct complexity of image sequence. The experimental results show that the proposed metric is more valid than other metrics, such as sequence correlation (SC) and interframe change degree (IFCD), and it is highly relevant to the actual performance of automatic target tracking algorithms.

红外单帧图像弱小目标检测技术综述

红外单帧图像弱小目标检测技术综述

Research on Boundary Recognition and Extraction Method of Field Operation Area based on UAV Remote Sensing Images

Precision agriculture is a mode of modern agricultural production and management on the basis of information technology, and it is an important way to achieve low consumption, high efficiency, fine quality and safety in agriculture. Moreover, accurate recognition and extraction of field operation area (FOA) boundary is a crucial basic data to implement precision agriculture. Due to the irregular shape, different planting ways and inconsistent size of FOA, it is difficult to recognize and extract the boundary. Therefore, the boundary of FOA based on UAV remote sensing images was classified in this paper, and a priori rule was proposed according to the boundary characteristics. Combined with the classical algorithm of image processing, a boundary recognition and extraction system was developed by LabVIEW to obtain the effective boundary of FOA. At last, the boundary recognition method and the accuracy and real-time performance of the extraction system were tested, and the result shows the method and system can accurately recognize and extract the actual boundary of FOA, and they are adapted to three different types of boundaries and image resolutions. The system has good real-time performance, and the single-frame image processing time does not exceed 100ms when the image resolution is lower than 1920⁎1280.

Facial expression recognition from image based on hybrid features understanding

Facial expression recognition (FER) plays an important role in the applications of human computer interaction. Given the wide use of convolutional neural networks (CNNs) in automatic video and image classification systems, higher-level features can be automatically learned from hierarchical neural networks with big data. However, learning CNNs require large amount of training data for adequate generalization, while the Scale-invariant feature transform (SIFT) does not need large training samples to generate useful feature. In this paper, we propose a new hybrid feature representation for the recognition of facial expressions from a single image frame that uses a combination of SIFT and deep-learning feature of different level extracted from the CNN model, then adopt the combined features and classify the expression by support vector machines (SVM). The performance of the proposed method has been validated on public CK+ databases. To evaluate the generalization ability of our method, we also performed an experiment on a cross-database environment. Experimental results show that the proposed approach can achieve better classification rates compared with state-of-art CNN methods, which indicate the considerable potential of combining shallow feature with deep-learning feature.

Background modeling method based on multi-feature fusion

In order to build a robust background model and improve the accuracy of detection of foreground objects, the temporal correlation of pixels at the same position of the video image and the spatial correlation of neighboring pixels are considered comprehensively. This paper proposed a background modeling method based on multi-feature fusion. By using the domain correlation of pixels in a single frame image to quickly establish an initial background model whichis updated using pixel values, frequency, update time and sensitivity of the video image sequence, the ghost phenomenon is effectively improved and the holes and false prospects for moving targets are reduced. Through multiple sets of data tests, it shows that the algorithm improves the adaptability and robustness of dynamic background and complex background.

A novel intensity-based multi-level classification approach for coronary plaque characterization in intravascular ultrasound images

BackgroundIntravascular ultrasound (IVUS) is a commonly used diagnostic imaging method for coronary artery disease. Virtual histology (VH) characterizes the plaque components into fibrous tissue (FT), fibro-fatty tissue (FFT), necrotic core (NC), or dense calcium (DC). However, VH can obtain only a single-frame image in one cardiac cycle, and specific software is needed to obtain the radio frequency data. This study proposed a novel intensity-based multi-level classification model for plaque characterization.MethodsThe plaque-containing regions between the intima and the media-adventitia were segmented manually for all IVUS frames. A total of 54 features including first order statistics, grey level co-occurrence matrix, Law’s energy measures, extended grey level run length matrix, intensity, and local binary pattern were estimated from the plaque-containing regions. After feature extraction, optimal features were selected using principle component analysis (PCA), and these were utilized as the input for the classification models. Plaque components were classified into FT, FFT, NC, or DC using an intensity-based multi-level classification model consisting of three different nets. Net 1 differentiated low-intensity components into FT/FFT and NC/DC groups. Then, net 2 subsequently divided FT/FFT into FT or FFT, whereas the remainder and high-intensity components were classified into NC or DC via net 3. To improve classification accuracy, each net utilized three different input features obtained by PCA. Classification performance was evaluated in terms of sensitivity, specificity, accuracy, and receiver operating characteristic curve.ResultsQuantitative results indicated that the proposed method showed significantly high classification accuracy for all tissue types. The classifiers had classification accuracies of 85.1%, 71.9%, and 77.2%, respectively, and the areas under the curve were 0.845, 0.704, and 0.783. In particular, the proposed method achieved relatively high sensitivity (82.0%) and specificity (87.1%) for differentiating between the FT/FFT and NC/DC groups.ConclusionsThese results confirmed the clinical applicability of the proposed approach for IVUS-based tissue characterization.

Infrared small-target detection under complex background based on subblock-level ratio-difference joint local contrast measure

It is always a challenging task to detect an infrared (IR) small target with high-detection rate, low false alarm rate, and high detection speed since the target usually has a small size and dim gray value, and the background is usually complex. Local contrast, which is based on the human visual system, has been proved efficient for IR small target detection, but existing local contrast algorithms are either difference-form or ratio-form and cannot enhance true target and suppress background simultaneously. In addition, most of them are pixel-level algorithms and require a huge amount of calculations. A simple but efficient method named subblock-level ratio-difference joint local contrast measure (SRDLCM) is proposed; it can enhance real small target and suppress complex background simultaneously. In addition, SRDLCM is calculated for each subblock but not each pixel, so its calculation amount can be reduced significantly. The experimental results on seven real IR sequences and one single-frame image dataset show that the proposed algorithm can achieve a good detection performance in detection rate and false alarm rate with a good robustness, and the time consumed for a single frame is only less than 0.04 s.