Pixel Motion Research Articles

Full parameters inversion model for mining subsidence prediction using simulated annealing based on single line of sight D-InSAR

Due to the inability of the single line of sight D-InSAR to monitor the three-dimensional deformation of the surface, the conventional methods are unable to obtain the prediction parameters (probability integral parameters) of surface subsidence in coal mining. In this paper, a calculation method of simulated annealing (SA) for probability integral parameters based on single line of sight D-InSAR is proposed. Firstly, the method predicts the subsidence, the horizontal movement in the north–south direction and the horizontal movement in the east–west direction of the target pixel by using the probability integral method. Based on the projection relationship between the three-dimensional deformation and the LOS deformation, the predicted movement and deformation of the target pixel in LOS direction ( $$r_{{i{\text{LOS}}}}^{'}$$ ) are calculated. Using the measured movement and deformation of the target pixel in LOS direction ( $$r_{{i{\text{LOS}}}}$$ ), the residuals of the target pixel are calculated ( $$v_{i} = r_{{i{\text{LOS}}}} - r_{{i{\text{LOS}}}}^{'}$$ ) and the error function of the parameter is constructed ( $$\varepsilon (B) = \sum {|v_{i} |}$$ ). Then based on the criteria ( $$\varepsilon (B) = \hbox{min}$$ ), all the probability integral parameters are obtained accurately by the SA method. The accuracy and robustness of the proposed method are verified by simulation experiments. At last, the predicted parameters of mining subsidence in 9310 working face of Nantun Coal Mine are calculated by this method, and the characteristics of probability integral parameters are analyzed.

Visual-inertial displacement sensing using data fusion of vision-based displacement with acceleration

In recognition of the importance of the displacement associated with assessing structural condition, many displacement measurement methods have been proposed to date. With advances in optics and electronics, displacement measurement relying on computer-vision techniques to convert pixel movement into structural displacement has drawn much attention recently, thanks to its simplicity in installation and relatively inexpensive cost. Despite numerous advantages, 2 major obstacles that prohibit the use of vision-based method are (a) resolution, which is a function of distance between the camera and the structure, and (b) limited frame rate, which both lower dynamic displacement-capturing capability. In this paper, to enhance the quality of vision-based displacement measurement, data fusion with acceleration measurement is proposed to improve the dynamic range of displacements while lowering signal noise. To achieve fusion between vision-based displacement and acceleration, complementary filters and a time synchronization method between 2 different sources were proposed. The proposed methods were verified through numerical analysis and an experimental test, the results of which showed the validity of proposed data fusion.

Depth recovering method immune to projector errors in fringe projection profilometry by use of cross-ratio invariance

In fringe projection profilometry, the projector parameters are difficult to calibrate accurately thus inducing errors in measurement results. To solve this problem, this paper analyzes the epipolar geometry of the fringe projection system, revealing that, on an epipolar plane, the depth variation along an incident ray induces the pixel movement along the epipolar line on the image plane of the camera. The depth variation and the pixel movement can be related to each other by using projective transformations. Under this condition, their cross-ratio keeps invariant. By use of this cross-ratio invariance, we suggest a depth recovering method immune to projector errors. To calibrate the measurement system, we shift a reference board perpendicularly to three positions with known depths and measure its phase maps as the reference phase maps. When measuring an object, we calculate the object depth at each pixel by equating the cross-ratio of the depths to that of the corresponding pixels having the same phase. Experimental results demonstrate that, with this technique, the errors associated with the projector, including its errors in geometry parameters, its lens distortions, and its luminance nonlinearity, do not affect the measurement results.

Discrete Sine Transform-Based Interpolation Filter for Video Compression

Fractional pixel motion compensation in high-efficiency video coding (HEVC) uses an 8-point filter and a 7-point filter, which are based on the discrete cosine transform (DCT), for the 1/2-pixel and 1/4-pixel interpolations, respectively. In this paper, discrete sine transform (DST)-based interpolation filters (DST-IFs) are proposed for fractional pixel motion compensation in terms of coding efficiency improvement. Firstly, a performance of the DST-based interpolation filters (DST-IFs) using 8-point and 7-point filters for the 1/2-pixel and 1/4-pixel interpolations is compared with that of the DCT-based IFs (DCT-IFs) using 8-point and 7-point filters for the 1/2-pixel and 1/4-pixel interpolations, respectively, for fractional pixel motion compensation. Finally, the DST-IFs using 12-point and 11-point filters for the 1/2-pixel and 1/4-pixel interpolations, respectively, are proposed only for bi-directional motion compensation in terms of the coding efficiency. The 8-point and 7-point DST-IF methods showed average Bjøntegaard Delta (BD)-rate reductions of 0.7% and 0.3% in the random access (RA) and low delay B (LDB) configurations, respectively, in HEVC. The 12-point and 11-point DST-IF methods showed average BD-rate reductions of 1.4% and 1.2% in the RA and LDB configurations for the Luma component, respectively, in HEVC.

Automated recording of home cage activity and temperature of individual rats housed in social groups: The Rodent Big Brother project.

Measuring the activity and temperature of rats is commonly required in biomedical research. Conventional approaches necessitate single housing, which affects their behavior and wellbeing. We have used a subcutaneous radiofrequency identification (RFID) transponder to measure ambulatory activity and temperature of individual rats when group-housed in conventional, rack-mounted home cages. The transponder location and temperature is detected by a matrix of antennae in a baseplate under the cage. An infrared high-definition camera acquires side-view video of the cage and also enables automated detection of vertical activity. Validation studies showed that baseplate-derived ambulatory activity correlated well with manual tracking and with side-view whole-cage video pixel movement. This technology enables individual behavioral and temperature data to be acquired continuously from group-housed rats in their familiar, home cage environment. We demonstrate its ability to reliably detect naturally occurring behavioral effects, extending beyond the capabilities of routine observational tests and conventional monitoring equipment. It has numerous potential applications including safety pharmacology, toxicology, circadian biology, disease models and drug discovery.

NIC: A Robust Background Extraction Algorithm for Foreground Detection in Dynamic Scenes

This paper presents a robust foreground detection method capable of adapting to different motion speeds in scenes. A key contribution of this paper is the background estimation using a proposed novel algorithm, neighbor-based intensity correction (NIC), that identifies and modifies the motion pixels from the difference of the background and the current frame. Concretely, the first frame is considered as an initial background that is updated with the pixel intensity from each new frame based on the examination of neighborhood pixels. These pixels are formed into windows generated from the background and the current frame to identify whether a pixel belongs to the background or the current frame. The intensity modification procedure is based on the comparison of the standard deviation values calculated from two pixel windows. The robustness of the current background is further measured using pixel steadiness as an additional condition for the updating process. Finally, the foreground is detected by the background subtraction scheme with an optimal threshold calculated by the Otsu method. This method is benchmarked on several well-known data sets in the object detection and tracking domain, such as CAVIAR 2004, AVSS 2007, PETS 2009, PETS 2014, and CDNET 2014. We also compare the accuracy of the proposed method with other state-of-the-art methods via standard quantitative metrics under different parameter configurations. In the experiments, NIC approach outperforms several advanced methods on depressing the detected foreground confusions due to light artifact, illumination change, and camera jitter in dynamic scenes.

Optical Tracking and Digital Quantification of Beating Behavior in Bioengineered Human Cardiac Organoids.

Organoid and organ-on-a-chip technologies are rapidly advancing towards deployment for drug and toxicology screening applications. Liver and cardiac toxicities account for the majority of drug candidate failures in human trials. Liver toxicity generally produces liver cell death, while cardiac toxicity causes adverse changes in heart beat kinetics. In traditional 2D cultures, beating kinetics can be measured by electrode arrays, but in some 3D constructs, quantifying beating kinetics can be more challenging. For example, real time measurements of calcium flux or contractile forces are possible, yet rather complex. In this communication article, we demonstrate a simple sensing system based on software code that optically analyzes video capture files of beating cardiac organoids, translates these files in representations of moving pixels, and quantifies pixel movement activity over time to generate beat kinetic plots. We demonstrate this system using bioengineered cardiac organoids under baseline and drug conditions. This technology offers a non-invasive, low-cost, and incredibly simple method for tracking and quantifying beating behavior in cardiac organoids and organ-on-a-chip systems for drug and toxicology screening.

Moving Object Detection With a Freely Moving Camera via Background Motion Subtraction

Detection of moving objects in a video captured by a freely moving camera is a challenging problem in computer vision. Most existing methods often assume that the background (BG) can be approximated by dominant single plane/multiple planes or impose significant geometric constraints on BG, or utilize a complex BG/foreground probabilistic model. Instead, we propose a computationally efficient algorithm that is able to detect moving objects accurately and robustly in a general 3D scene. This problem is formulated as a coarse-to-fine thresholding scheme on the particle trajectories in the video sequence. First, a coarse foreground (CFG) region is extracted by performing reduced singular value decomposition on multiple matrices that are built from bundles of particle trajectories. Next, the BG motion of pixels in the CFG region is reconstructed by a fast inpainting method. After subtracting the BG motion, the fine foreground is segmented out by an adaptive thresholding method that is capable of solving multiple-moving-objects scenarios. Finally, the detected foreground is further refined by the mean-shift segmentation method. Extensive simulations and a comparison with the state-of-the-art methods verify the effectiveness of the proposed method.

Appearance-Based Gaze Estimation via Uncalibrated Gaze Pattern Recovery.

Aiming at reducing the restrictions due to person/scene dependence, we deliver a novel method that solves appearance-based gaze estimation in a novel fashion. First, we introduce and solve an "uncalibrated gaze pattern" solely from eye images independent of the person and scene. The gaze pattern recovers gaze movements up to only scaling and translation ambiguities, via nonlinear dimension reduction and pixel motion analysis, while no training/calibration is needed. This is new in the literature and enables novel applications. Second, our method allows simple calibrations to align the gaze pattern to any gaze target. This is much simpler than conventional calibrations which rely on sufficient training data to compute person and scene-specific nonlinear gaze mappings. Through various evaluations, we show that: 1) the proposed uncalibrated gaze pattern has novel and broad capabilities; 2) the proposed calibration is simple and efficient, and can be even omitted in some scenarios; and 3) quantitative evaluations produce promising results under various conditions.

Out-of-home activity analysis using a low-resolution visual sensor

Loneliness and social isolation are probably the most prevalent psychosocial problems related to aging. One critical component in assessing social isolation in an unobtrusive manner is to measure the out-of-home activity levels, as social isolation often goes along with decreased physical activity, decreased motoric functioning, and a decline in activities of daily living, all of which may lead to a reduction in the amount of time spent out-of-home. In this work, we propose to use a single visual sensor for detecting out-of-home activity. The visual sensor has a very low spatial resolution (900 pixels), which is a key feature to ensure a cheap technology and to maintain the user’s privacy. Firstly, the visual sensor is installed in a top view setup at the door entrance. Secondly, a correlation-based foreground detection method is used to extract the foreground. Thirdly, an Extra Trees Classifier (ETC) is trained to classify the directionality of the person (in/out) based on the motion of the foreground pixels. Due to the nature of variability of the out-of-home activity, the relative frequency of the directionality (in/out) is measured over a window of 3 seconds to determine the final result. We installed our system in 9 different service flats in the UK, Belgium and France where the same ETC model is used. We evaluate our method on video sequences captured in real-life environments from the different setups, where the persons’ out-of-home routines are recorded. The results show that our approach of detecting out-of-home activity achieves an accuracy of 91.30%.

Video Motion Vector Recovery Method Using Decoding Partition Information

This paper presents a novel motion vector recovery and error concealment algorithm with the utilization of encoding partition information for H.264/AVC. The motion vectors for each missing pixel location are derived using available neighboring pixel motion vectors, which contribute to the generation of the missing motion vectors inversely proportional to the distance between them. The motion extrapolation method is used to project the encoding partition information from the reference frame into the current frame with different levels of overlapping of lost pixels. The different levels of overlapping can help determine the estimated encoding partition information in the lost macroblock (MB). Finally, the pixels that are determined to be of the same estimated partition share the same motion vector in order to maintain the integrity of the estimated moving objects in the lost MB. This proposed pixel-based motion vector with partition (PMVP) method compares with the state-of-the-art Zhou's method, Lin's method, and Lie's method. For total average in packet loss rates of 3%, 7%, 16%, and 20%, PMVP is better than Zhou by 0.88, 1.02, 1.05, and 1.01 dB, respectively; Lin by 0.22, 0.32, 0.35, and 0.33 dB, respectively; and Lie by 4.12, 4.98, 4.15, and 3.88 dB, respectively. Therefore, the proposed PMVP performs the best on average among all the methods.

Survey on Algorithm and VLSI Architecture for MPEG-Like Video Coder

Efficient and dedicated hardware architecture and accelerator micro-engines are crucial implementation forms of MPEG-like video coder. It is significant to excavate and generalize the common technologies and design philosophy of hardwired MPEG-like coders behind number of architectures from academic and industrial communities. This paper makes systematic survey on algorithm and architecture of hardwired MPEG-like coders, from microscopic and macroscopic perspectives, taking H.264/AVC as the analysis target. Recent advances in hardware architectures of prevailing H.264/AVC coders are reviewed and summarized. Furthermore, important algorithm modules, such as integer and fractional pixel motion estimation, mode decision, motion vector prediction, intra prediction, rate control, CABAC coder and deblocking filter are reviewed with detailed analysis on algorithm and hardware architecture. In accordance with the intrinsic characteristics of the algorithm flows, the major design constraints and consideration factors of algorithm and architecture are analyzed respectively. The common technologies of the prevailing architectures are summarized from a systematic perspective, coving different levels ranging from algorithm, architecture, to control and data flows, etc. Based on these analysis, this survey further highlights in-depth summarization and perspectives on MPEG-like coder architecture design. First, the design challenges with multiple target performance optimization are analyzed, and the possible solutions for design challenges are systematically summarized. Second, the rate-distortion-complexity constrained algorithm optimization for MPEG-like video encoder is discussed. Third, typical four-level hierarchical architecture model (SoC system, module, inter-connection, memory) is analyzed, and the pivotal memory architecture and inter-connection architecture are emphasized for analysis. Moreover, the algorithm and architecture design suggestions and preferences for the vital modules are discussed. Fourth, the composite performances of prevailing architectures are evaluated. The concerned target parameters including hardware logic cost, SRAM size, external memory bandwidth, throughput efficiency, power dissipation, and rate-distortion performance are taken as comparison factors. Finally, this paper provides explicit perspectives on future trends of video coder architecture design. The proposed paper can be taken as design reference for H.264/AVC coder hardware architecture, and offer further insight into algorithm and architecture optimization for the new emerging HEVC standard.

Foreground Segmentation for Live Videos by Texture Features

This paper presents a method to extract the foreground images from live videos by means of automatic object segmentation. The parameters such as colour, motion of the pixel and image texture or more specifically texture constraints are used for segmentation. A cellular neural network which combines both colour as well as motion of pixels which varies from frame to frame is implemented which helps in accurately separating the boundaries and thus reducing misclassifications. The global motion of pixels is calculated by computing the forward and backward displaced frame differences (DFDs) with the respect to the current frame. The texture constraint for each pixel to be labeled is calculated from the difference between their corresponding texture descriptors and the texture prior models which is provided by the Local Binary Pattern (LBP) histogram. Finally by means of the randomized texton searching algorithm and graph cut frame work the foreground is extracted from the video.

2D to 3D Conversion using Key Frame Extraction

Background/Objectives: Extracting 3D information from two dimensional images is tedious process and challenging task. In this paper, a novel 2D to 3D conversion model is presented to convert monoscopic 2D images into 3D stereoscopic images. Method/Satistical Analysis: The depth information about key frames is extracted from the 2D images. Then both foreground and background objects are extracted using background subtraction algorithm. From the generated region of interest both forward and backward motion pixels are extracted in the form of vectors. Gabor filter is applied to decompose high pass luminance and chrominance. Each pixels decomposed with gabor filter is estimated with its own sub band to model the depth information. Findings: Depth map associated with each sub band and its oriented pixels are mapped with the depth information of 2D images and this makes a realistic view of 2D images in 3D stereoscopic view. Application/ Improvements: The proposed model is experimentally tested on both right and left view and the results shows that the presented model is better than the traditional model.

Motion-based skin region of interest detection with a real-time connected component labeling algorithm

This paper presents a motion-based skin Region of Interest (ROI) detection method using a real-time connected component labeling algorithm to provide real-time and adaptive skin ROI detection in video images. Skin pixel segmentation in video images is a pre-processing step for face and hand gesture recognition, and motion is a cue for detecting foreground objects. We define skin ROIs as pixels of skin-like color where motion takes place. In the skin color estimation phase, RGB color histograms are utilized to define the skin color distribution and specify the threshold to segment skin-like regions. A parallel computed connected component labeling algorithm is also proposed to group the segmentation results into several clusters. If a cluster covers any motion pixel, this cluster is identified as a skin ROI. The method's results for real images are shown, and its speed is evaluated for various parameters. This technology is compatible with monitoring systems, scene understanding, and natural user interfaces.

Motion Based Animal Detection in Aerial Videos

Computer vision techniques are applied to perform automatic wildlife surveying and animal monitoring. Animal detection in aerial videos is challenging because of the complexity of wild environments. In this paper, a method for moving animal detection is proposed by taking advantage of global patterns of pixel motion. In the video dataset, where animals make obvious movement against the background, motion vectors of each pixel are estimated by applying optical flow methods. A coarse segmentation then removes most parts of the background by applying a pixel velocity threshold. Based on the segmented regions, another threshold was employed to filter out negative candidates that could belong to the background. The pros and cons of this method are discussed.

Image Registration using combination of PCA and GPOF Method for Multiframe Super-Resolution

Super-resolution technique is used for resolution enhancement. In Multiframe Super-resolution multiple low resolution images are combined to get high resolution image. Most important part of Multiframe Super-resolution is Image Registration that estimates translation,rotation and scaling parameters. In this paper they propose combination of PCA and GPOF registration method for constructing high resolution image. In GPOF registration method, which reach the sub pixel precision and allow large pixel motions and then apply PCA method which is used for compression of image. Image fusion is applied to get one output image. Then bicubic interpolation is used to get high resolved image. Experiment images show that the HR image by our proposed method has much higher quality than other methods.

Moment feature based fast feature extraction algorithm for moving object detection using aerial images.

Fast and computationally less complex feature extraction for moving object detection using aerial images from unmanned aerial vehicles (UAVs) remains as an elusive goal in the field of computer vision research. The types of features used in current studies concerningmoving object detection are typically chosen based on improving detection rate rather than on providing fast and computationally less complex feature extraction methods. Because moving object detection using aerial images from UAVs involves motion as seen from a certain altitude, effective and fast feature extraction is a vital issue for optimum detection performance. This research proposes a two-layer bucket approach based on a new feature extraction algorithm referred to as the moment-based feature extraction algorithm (MFEA). Because a moment represents thecoherent intensity of pixels and motion estimation is a motion pixel intensity measurement, this research used this relation to develop the proposed algorithm. The experimental results reveal the successful performance of the proposed MFEA algorithm and the proposed methodology.

Efficient assessment method of on-board modulation transfer function of optical remote sensing sensors.

Modulation transfer function (MTF) can be used to evaluate the imaging performance of on-board optical remote sensing sensors, as well as recover and restore images to improve imaging quality. Laboratory measurement approaches for MTF have achieved high precision. However, they are not yet suitable for on-board measurement. In this paper, a new five-step approach to calculate MTF of space optical remote sensing sensors is proposed. First, a pixel motion model is used to extract the conditional sub-frame images. Second, a mathematical morphology algorithm and a correlation-homomorphic filter algorithm are used to eliminate noise and enhance sub-frame image. Third, an image partial differentiation determines the accurate position of edge points. Fourth, a model optical function is used to build a high-resolution edge spread function. Finally, MTF is calculated by derivation and Fourier transform. The experiment shows that the assessment method of MTF is superior to others.

DSP Embedded Early Fire Detection Method Using IR Thermal Video

Here we present a simple flame detection method for an infrared (IR) thermal camera based real-time fire surveillance digital signal processor (DSP) system. Infrared thermal cameras are especially advantageous for unattended fire surveillance. All-weather monitoring is possible, regardless of illumination and climate conditions, and the data quantity to be processed is one-third that of color videos. Conventional IR camera-based fire detection methods used mainly pixel-based temporal correlation functions. In the temporal correlation function-based methods, temporal changes in pixel intensity generated by the irregular motion and spreading of the flame pixels are measured using correlation functions. The correlation values of non-flame regions are uniform, but the flame regions have irregular temporal correlation values. To satisfy the requirement of early detection, all fire detection techniques should be practically applied within a very short period of time. The conventional pixel-based correlation function is computationally intensive. In this paper, we propose an IR camera-based simple flame detection algorithm optimized with a compact embedded DSP system to achieve early detection. To reduce the computational load, block-based calculations are used to select the candidate flame region and measure the temporal motion of flames. These functions are used together to obtain the early flame detection algorithm. The proposed simple algorithm was tested to verify the required function and performance in real-time using IR test videos and a real-time DSP system. The findings indicated that the system detected the flames within 5 to 20 seconds, and had a correct flame detection ratio of 100% with an acceptable false detection ratio in video sequence level.