Grayscale Image Sequences Research Articles

Head tracking using an optical soft tactile sensing surface.

This research proposes a sensor for tracking the motion of a human head via optical tactile sensing. It implements the use of a fibrescope a non-metal alternative to a webcam. Previous works have included robotics grippers to mimic the sensory features of human skin, that used monochrome cameras and depth cameras. Tactile sensing has shown advantages in feedback-based interactions between robots and their environment. The methodology in this paper is utilised to track motion of objects in physical contact with these sensors to replace external camera based motion capture systems. Our immediate application is related to detection of human head motion during radiotherapy procedures. The motion was analysed in two degrees of freedom, respective to the tactile sensor (translational in z-axis, and rotational around y-axis), to produce repeatable and accurate results. The movements were stimulated by a robot arm, which also provided ground truth values from its end-effector. The fibrescope was implemented to ensure the device's compatibility with electromagnetic waves. The cameras and the ground truth values were time synchronised using robotics operating systems tools. Image processing methods were compared between grayscale and binary image sequences, followed by motion tracking estimation using deterministic approaches. These included Lukas-Kanade Optical Flow and Simple Blob Detection, by OpenCV. The results showed that the grayscale image processing along with the Lukas-Kanade algorithm for motion tracking can produce better tracking abilities, although further exploration to improve the accuracy is still required.

DEF: Deep Ensemble Neural Network Classifier for Android Malware Detection

Malware is one of the threats to security of computer networks and information systems. Since malware instances are available sufficiently, there is increased interest among researchers on usage of Artificial Intelligence (AI). Of late AI-enabled methods such as machine learning (ML) and deep learning paved way for solving many real-world problems. As it is a learning-based approach, accumulated training samples help in improving thequality of training and thus leveraging malware detection accuracy. Existing deep learning methods are focusing on learning-based malware detection systems. However, there is need for improving the state of the art through ensemble approach. Towards this end, in this paper we proposed a framework known as Deep Ensemble Framework (DEF) for automatic malware detection. The framework obtains features from training samples. From given malware instance a grayscale image is generated. There is another process to extract the opcode sequences. Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) techniques are used to obtain grayscale image and opcode sequence respectively. Afterwards, a stacking ensemble is employed in order to achieve efficient malware detection and classification. Malware samples collected from the Internet sources and Microsoft are used for the empirical study. An algorithm known as Ensemble Learning for Automatic Malware Detection (EL-AML) is proposed to realize our framework. Another algorithm named Pre-Process is proposed to assist the ELAML algorithm for obtaining intermediate features required by CNN and LSTM. Empirical study reveals that our framework outperforms many existing methods in terms of speed-up and accuracy.

Respiratory motion correction for liver contrast-enhanced ultrasound by automatic selection of a reference image.

Respiratory motion correction is necessary for the quantitative analysis of liver contrast-enhanced ultrasound (CEUS) image sequences. Most respiratory motion correction methods are based on the dual mode of CEUS image sequences, including contrast and grayscale image sequences. Due to free-breathing motion, the acquired two-dimensional (2D) ultrasound cine might show the in-plane and out-of-plane motion of tumors. The registration of an entire 2D ultrasound contrast image sequence based on out-of-plane images is ineffective. For the respiratory motion correction of CEUS sequences, the reference image is usually considered the standard for the deletion of any out-of-plane images. Most methods used for the selection of the reference image are subjective in nature. Here, a quantitative selection method for an optimal reference image from CEUS image sequences in the B mode and contrast mode was explored. The original high-dimensional ultrasound grayscale image data were mapped into a two-dimensional space using Laplacian Eigenmaps (LE), and K-means clustering was adopted. The center image of the larger cluster with a near-peak contrast intensity was considered the optimal ultrasound reference image. In the ultrasound grayscale image sequence, the images with the maximum correlations to the reference image in the same time interval were selected as the corrected image sequence. The effectiveness of this proposed method was then validated on 18 CEUS cases of VX2 tumors in rabbit livers. Correction smoothed the time-intensity curves (TICs) extracted from the region of interest of the CEUS image sequences. Before correction, the average of the total mean structural similarity (TMSSIM) and the average of the mean correlation coefficient (MCC) from the image sequences were 0.45±0.11 and 0.67±0.16, respectively, and after correction, the average TMSSIM and MCC increased (P<0.001) by 31% to 0.59±0.11 and by 21% to 0.81±0.11, respectively. The average deviation value (DV) index of the TICs from the image sequences prior to correction was 92.16±18.12, and correction reduced the average to 31.71±7.31. The average TMSSIM and MCC values after correction using the mean frame of the reference image (MBMFRI) were clearly lower than those after correction using the proposed method (P<0.001). Moreover, the average DV after correction using the MBMFRI was obviously higher than that after correction using the proposed method (P<0.001). The breathing frequency of rabbits is notably faster than that of human beings, but the proposed correction method could reduce the effect of the respiratory motion in the CEUS image sequences. The reference image was selected quantitatively, which could improve the accuracy of the quantitative analysis of rabbit liver CEUS sequences using the reference image method based on the current standard of manual selection and the MBMFRI. Thiseasy-to-operatemethod can potentially be used in both animal studies and clinical applications.

DICOM Calibration of Commercial Off-The-Shelf (COTS) display devices for use in radiotherapy

In a modern, distributed radiotherapy environment image evaluation is becoming increasingly common at office desktop computers, many of which will be equipped with inexpensive Commercial Off-The-Shelf (COTS) display devices. To ensure consistency of image interpretation it is important to achieve consistency of display device performance. 15 displays (4 different models from 2 manufacturers) were characterised and the potential for their calibration against the DICOM Greyscale Standard Display Function (GSDF) explored. Each display’s luminance output was measured as a function of grey-level using a calibrated LXChroma photometer (IBA Dosimetry, Germany) with an automated sequence of greyscale images. A modification to the graphics card look-up table was calculated to adjust system performance to match the GSDF and the measurements were repeated. Prior to calibration there was considerable variation in performance across displays, including between those of the same model and manufacturer. The max luminance and max/min luminance ratio ranged from 94–401 cd/m2 (m = 213 cd/m2, sd = 95 cd/m2) and 85–1005 (m = 228, sd = 223) respectively. Initially, no display matched the GSDF to within the ‘secondary class’ limit of ±20% across all light levels, with the error being between 0.2 and 63.0%. Following calibration, all displays but one were within ±15% of the GSDF, with the outlier being +21% at a single measurement point. Care must be taken when deploying COTS displays because of initial variability in performance. However, performance can be successfully standardised through calibration against the GSDF. When appropriately calibrated, high quality yet relatively inexpensive COTS displays may offer a suitable alternative to traditional, expensive ‘medical grade’ display devices.

Detecting Malware with an Ensemble Method Based on Deep Neural Network

Malware detection plays a crucial role in computer security. Recent researches mainly use machine learning based methods heavily relying on domain knowledge for manually extracting malicious features. In this paper, we propose MalNet, a novel malware detection method that learns features automatically from the raw data. Concretely, we first generate a grayscale image from malware file, meanwhile extracting its opcode sequences with the decompilation tool IDA. Then MalNet uses CNN and LSTM networks to learn from grayscale image and opcode sequence, respectively, and takes a stacking ensemble for malware classification. We perform experiments on more than 40,000 samples including 20,650 benign files collected from online software providers and 21,736 malwares provided by Microsoft. The evaluation result shows that MalNet achieves 99.88% validation accuracy for malware detection. In addition, we also take malware family classification experiment on 9 malware families to compare MalNet with other related works, in which MalNet outperforms most of related works with 99.36% detection accuracy and achieves a considerable speed-up on detecting efficiency comparing with two state-of-the-art results on Microsoft malware dataset.

Continuous detection of human fall using multimodal features from Kinect sensors in scalable environment

Background and Objectives: Automatic detection of human fall is a key problem in video surveillance and home monitoring. Existing methods using unimodal data (RGB / depth / skeleton) may suffer from the drawbacks of inadequate lighting condition or unreliability. Besides, most of proposed methods are constrained to a small space with off-line video stream.Methods: In this study, we overcome these encountered issues by combining multi-modal features (skeleton and RGB) from Kinect sensor to take benefits of each data characteristic. If a skeleton is available, we propose a rules based technique on the vertical velocity and the height to floor plane of the human center. Otherwise, we compute a motion map from a continuous gray-scale image sequence, represent it by an improved kernel descriptor then input to a linear Support Vector Machine. This combination speeds up the proposed system and avoid missing detection at an unmeasurable range of the Kinect sensor. We then deploy this method with multiple Kinects to deal with large environments based on client server architecture with late fusion techniques.Results: We evaluated the method on some freely available datasets for fall detection. Compared to recent methods, our method has a lower false alarm rate while keeping the highest accuracy. We also validated on-line our system using multiple Kinects in a large lab-based environment. Our method obtained an accuracy of 91.5% at average frame-rate of 10fps.Conclusions: The proposed method using multi-modal features obtained higher results than using unimodal features. Its on-line deployment on multiple Kinects shows the potential to be applied in to any of living space in reality.

Compensation of in-plane rigid motion for in vivo intracoronary ultrasound image sequence

Intracoronary ultrasound (ICUS) is an interventional imaging modality that is used to acquire a series of tomographic images from the vascular lumen, for diagnosis and treatment of coronary artery diseases in clinical settings. Motion artifacts caused by cardiac dynamics and the pulsatile blood flow within the vascular lumen, during continuous pullback (non-gated) ICUS image acquisition, hinder visualization of longitudinal cuts, assessment of arterial morphology and hemodynamics, and three-dimensional (3-D) vessel reconstruction. The aim of this study is to develop a method to compensate for in-plane rigid motion in non-gated in vivo ICUS sequences. The signals associated with cardiac motion are first detected from the gray-scale image sequence. They are represented with rigid motion parameters between luminal contours extracted from successive slices. Subsequently, the signals were filtered to separate the dynamic components caused by cardiac motion from those caused by the irregular morphology of the vascular lumen. Dynamic components were then compensated by performing a back transformation of related pixels within the vessel region in each frame. The method is validated by computer-simulation and using real ICUS image data. Possible sources of error are discussed based on the experimental results.

Computing Flow Field for Colorizing Video Clips

Although there are many effects that can be added to the images or movies for their development, this paper deals with coloring grayscale video clip because it is an active and challenging area of research. In addition, colorization has become a very profitable business. Colorization of video clip requires tracking regions across the frames of a shot. Existing tracking algorithms typically fail to robustly track non-rigid regions, requiring massive user intervention in the process. In order to reduce the time and effort required of the user to colorize video clips, we propose semi-automatic video colorization by propagating the colors from key frame to the rest of the sequences of grayscale images by computing flow field (optical flow) between the images. Key frame is a color image that is used to propagate colors to the rest of the sequence of grayscale images. Experimental results show the algorithm works quite well with good-quality results.

Simultaneous fusion, compression, and encryption of multiple images

We report a new spectral multiple image fusion analysis based on the discrete cosine transform (DCT) and a specific spectral filtering method. In order to decrease the size of the multiplexed file, we suggest a procedure of compression which is based on an adapted spectral quantization. Each frequency is encoded with an optimized number of bits according its importance and its position in the DC domain. This fusion and compression scheme constitutes a first level of encryption. A supplementary level of encryption is realized by making use of biometric information. We consider several implementations of this analysis by experimenting with sequences of gray scale images. To quantify the performance of our method we calculate the MSE (mean squared error) and the PSNR (peak signal to noise ratio). Our results consistently improve performances compared to the well-known JPEG image compression standard and provide a viable solution for simultaneous compression and encryption of multiple images.

A fuzzy filter for the removal of random impulse noise in image sequences

In this paper a new fuzzy filter for the removal of random impulse noise in digital grayscale image sequences is presented. The filter consists of different noise detection and filtering steps, in which the fuzzy set theory is used. This noise detection is based both on spatial and on temporal information and has the aim to prevent the filtering of noise free image pixels. The filtering of the detected noisy pixels is finally performed in a motion compensated way. Experimental results show that our method outperforms other state-of-the-art filters in terms of the peak-signal-to-noise ratio as well as visual quality.

Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking

Appearance modeling is very important for background modeling and object tracking. Subspace learning-based algorithms have been used to model the appearances of objects or scenes. Current vector subspace-based algorithms cannot effectively represent spatial correlations between pixel values. Current tensor subspace-based algorithms construct an offline representation of image ensembles, and current online tensor subspace learning algorithms cannot be applied to background modeling and object tracking. In this paper, we propose an online tensor subspace learning algorithm which models appearance changes by incrementally learning a tensor subspace representation through adaptively updating the sample mean and an eigenbasis for each unfolding matrix of the tensor. The proposed incremental tensor subspace learning algorithm is applied to foreground segmentation and object tracking for grayscale and color image sequences. The new background models capture the intrinsic spatiotemporal characteristics of scenes. The new tracking algorithm captures the appearance characteristics of an object during tracking and uses a particle filter to estimate the optimal object state. Experimental evaluations against state-of-the-art algorithms demonstrate the promise and effectiveness of the proposed incremental tensor subspace learning algorithm, and its applications to foreground segmentation and object tracking.

An intensive restraint topology adaptive snake model and its application in tracking dynamic image sequence

The aim of this work is to propose an improved intensive restraint topology adaptive snake model for detecting and tracking target contours with complex topological morphology from dynamic gray-scale image sequences. The image is discretized by orthogonal equidistant grid lines and snake nodes are constrained to move only from one grid vertex to another along grid lines. Thus, the computation is much simpler than those of other T-snakes since complicated calculation of intersection points of snake curve and grid lines is not required. Topological transformation is implemented through splitting snake nodes as well as automatically detecting and eliminating topological collision. An adaptive weighted expansion energy term is introduced into snake’s energy function driving the model to pass across the region with non-uniform intensity distribution inside the target region. The required manual interaction is reduced to only select a random point within the target region in the first frame of the image sequence. For subsequent frames, the detection result in the previous frame is regarded as the initial model in the current frame. Through experiments on simulated image data and clinical X-ray coronary angiographic sequences, the model’s topological flexibility and lower computation cost compared with other T-snakes are demonstrated. Also, the sensitivity of the model to weights of the energy terms and grid size as well as the initial plan of the model is discussed based on experimental results.

Photometric reconstruction of a dynamic textured surface from just one color image acquisition

Textured surface analysis is essential for many applications. We present a three-dimensional recovery approach for real textured surfaces based on photometric stereo. The aim is to be able to measure the textured surfaces with a high degree of accuracy. For this, we use a color digital sensor and principles of color photometric stereo. This method uses a single color image, instead of a sequence of gray-scale images, to recover the surface of the three dimensions. It can thus be integrated into dynamic systems where there is significant relative motion between the object and the camera. To evaluate the performance of our method, we compare it on real textured surfaces to traditional photometric stereo using three images. We thus show that it is possible to have similar results with just one color image.

COLOR INTENSITY PROJECTION OF DIGITALLY SUBTRACTED ANGIOGRAPHY FOR THE VISUALIZATION OF BRAIN ARTERIOVENOUS MALFORMATIONS

Reliable and rapid delineation of arteriovenous malformations enables the application of effective treatments such as stereotactic radiosurgery. We describe a new method to improve the speed and reliability of visualizing the flow of contrast images with digital subtraction angiography. In line with current practices, digital subtraction angiography was used to produce a sequence of grayscale images. The new method combines the standard grayscale images produced by digital subtraction angiography into a single composite color image that encodes the contrast arrival time at each point of the brain's circulatory system. The algorithm is simple, fast, and easy to implement. The technique allows the flow of contrast from a series of angiography images to be summarized in a single color image. This visualization method promises to improve the speed of manual delineation of arteriovenous malformations. Further studies are required to evaluate the clinical value of the use of color intensity projection images, supplemented by grayscale images as necessary, in comparison with contouring on grayscale images only.

Designing an Embedded Video Processing Camera Using a 16-bit Microprocessor for a Surveillance System

We describe the design and implementation of a hybrid intelligent surveillance system consisting of an embedded system and a personal computer (PC)-based system. The embedded system performs some of the image processing tasks and sends the processed data to a PC. The PC tracks persons and recognizes two-person interactions by using a grayscale side-view image sequence captured by a stationary camera. Based on our previous research, we explored the optimum division of tasks between the embedded system and the PC, simulated the embedded system using dataflow models in Ptolemy, and prototyped the embedded systems in real-time hardware and software using a 16-bit CISC microprocessor. This embedded system processes one frame image in 89 ms, which is within three frame-cycle periods for a 30 Hz video system. In addition, the real-time embedded system prototype uses 5.7 Kbytes of program memory, 854 Kbytes of internal data memory and 2 Mbytes external DRAM.

Robust Computation of Optical Flow using Orientation Code Matching.

A robust matching scheme for computing optical flow within a sequence of grayscale images is proposed. The technique employs the gradient information in textured images for extracting features in the form of orientation codes, which are then used for matching. The proposed method has been found to be robust in cases of matching under different ill-conditionings especially illumination variations. We utilize its robustness to compute optical flow in cases where illumination fluctuation is a problem and matching pixel brightness can introduce errors. Results of computation of optical flow field on real world scenes in the cases of translation, rotation and zooming have been presented and compared with other region matching techniques.

A novel method for tracking and counting pedestrians in real-time using a single camera

This paper presents a real-time system for pedestrian tracking in sequences of grayscale images acquired by a stationary camera. The objective is to integrate this system with a traffic control application such as a pedestrian control scheme at intersections. The proposed approach can also be used to detect and track humans in front of vehicles. Furthermore, the proposed schemes can be employed for the detection of several diverse traffic objects of interest (vehicles, bicycles, etc.) The system outputs the spatio-temporal coordinates of each pedestrian during the period the pedestrian is in the scene. Processing is done at three levels: raw images, blobs, and pedestrians. Blob tracking is modeled as a graph optimization problem. Pedestrians are modeled as rectangular patches with a certain dynamic behavior. Kalman filtering is used to estimate pedestrian parameters. The system was implemented on a Datacube MaxVideo 20 equipped with a Datacube Max860 and was able to achieve a peak performance of over 30 frames per second. Experimental results based on indoor and outdoor scenes demonstrated the system s robustness under many difficult situations such as partial or full occlusions of pedestrians.

Development of Vision-Based Vehicle Detection and Recognition System for Intelligent Vehicles

The business plan for the Intelligent Vehicles Initiative (IVI) from the U.S. Department of Transportation contains several candidate services that address collision warning and collision avoidance. Recently, a proactive crash mitigation system that would enhance the crash avoidance and survivability components of IVI has been proposed. An accurate object detection and recognition system is a prerequisite for the proactive crash mitigation system. A vision-based approach to the detection and recognition of vehicles, calculation of their motion parameters, and tracking of multiple vehicles by using a sequence of gray-scale images taken from a moving vehicle is presented. The vision-based system consists of four models: the object detection model, the object recognition model, the object information model, and the object tracking model. Object detection, recognition, and tracking are accomplished by combining the analysis of a single image frame with the analysis of consecutive image frames. In the analysis of the single image frame, the system detects the potential objects by using their shape features and recognizes the objects by using a neural network. Once the objects are recognized, they are tracked in the consecutive image frames by processing only the pertinent areas given by previous frames. The analysis of the single image frame is performed every 10 image frames. The information model will judge whether the objects are hazardous to the host vehicle by using two parameters: time to collision and its time derivative. Experimental results demonstrated a robust system in real-time vehicle recognition, vehicle tracking, and vehicle motion analysis over thousands of image frames.