Pixel Correspondence Research Articles

Physical activity recognition based on motion in images acquired by a wearable camera

A new technique to extract and evaluate physical activity patterns from image sequences captured by a wearable camera is presented in this paper. Unlike standard activity recognition schemes, the video data captured by our device do not include the wearer him/herself. The physical activity of the wearer, such as walking or exercising, is analyzed indirectly through the camera motion extracted from the acquired video frames. Two key tasks, pixel correspondence identification and motion feature extraction, are studied to recognize activity patterns. We utilize a multiscale approach to identify pixel correspondences. When compared with the existing methods such as the good features detector and the speed-up robust feature (SURF) detector, our technique is more accurate and computationally efficient. Once the pixel correspondences are determined which define representative motion vectors, we build a set of activity pattern features based on motion statistics in each frame. Finally, the physical activity of the person wearing a camera is determined according to the global motion distribution in the video. Our algorithms are tested using different machine learning techniques such as the K-nearest neighbor (KNN), naive Bayesian and support vector machine (SVM). The results show that many types of physical activities can be recognized from field acquired real-world video. Our results also indicate that, with a design of specific motion features in the input vectors, different classifiers can be used successfully with similar performances.

Stereo Image Retargeting with Shift-Map

We propose a new stereo image retargeting method based on the framework of shift-map image editing. Retargeting is the process of changing the image size according to the target display while preserving as much of the richness of the image as possible, and is often applied to monocular images and videos. Retargeting stereo images poses a new challenge because pixel correspondences between the stereo pair should be preserved to keep the scene's structure. The main contribution of this paper is integrating a stereo correspondence constraint into the retargeting process. Among several retargeting methods, we adopt shift-map image editing because this framework can be extended naturally to stereo images, as we show in this paper. We confirmed the effectiveness of our method through experiments.

３次元映像情報メディア技術 多視点デプスマップの情報を用いた高効率多視点映像符号化

We present an efficient coding method for multiview video plus a multiview depth map. This representation has been discussed as a potential standard for advanced visual media such as free-viewpoint TV and 3D video. The method is used to predict three kinds of information, disparity vectors, coding modes, and image signals, by utilizing pixel correspondences among views that can be calculated from coded multiview depth maps. Experiments show that using the proposed method reduces the bitrate by up to 22.5% relative to simulcast multiview video coding of multiview video and depth maps. The results also show the coding are not significantly affected by encoding noises on depth maps. Furthermore, the bitrate is reduced by up to 11.8% for multiview video coding where the bitrate for the depth maps is treated as only an overhead.

SU-GG-J-27: An Improved Demons Algorithm by Incorporating Accurate Voxel Motion Calculation

Purpose: The Demons algorithm is widely used for deformable registration of images from the same imaging modality by iteratively solving the gradient constraint equation to build pixel correspondence between the reference and float images. The algorithm itself does not solve the equation completely and it assumes that pixels move in the direction of the gradient, which is not true for most circumstances. We propose a more accurate optical flow or motion vector calculation method to improve the registration performance. Method and Materials: For a 3D CT-to-CT registration problem, the Demons algorithm simply uses the gradient at each voxel as the 3D motion vector since one gradient constraint equation has three unknown variables. We first apply the Lucas Kanade method to solve the gradient constraint equation by including more nearby voxels. For some voxels near the contours, the temporal derivatives between the two images are too large, causing unrealistic large motion vectors. Similar to the Demons algorithm, a weighting factor is proposed and added to the original resolution using the Lucas Kanade method. The performance of both algorithms is compared for real patient cases. The float image is generated by rigidly or non-rigidly transforming the reference image. The vector field generated after the registration is compared to the original transformation. The planning prostate CT and CT-on-rails images from the same patient are registered using both methods for comparison. A multi-resolution scheme is also applied to overcome large voxel shift between images. Results: Using known translation, rotation or non-rigid deformation fields, the results from this study show the proposed algorithm quantitatively outperforms the Demons algorithm. This is further confirmed by patient studies demonstrating better registration results for the proposed algorithm. Conclusion: An improved Demons algorithm is investigated that may have a great potential to improve the registration performance for image-guided radiation therapy.

ENCODING VIDEO SEQUENCES IN FRACTAL-BASED COMPRESSION

With the rapid increase in the use of computers and the Internet, the demand for higher transmission and better storage is increasing as well. This paper describes the different techniques for data (image-video) compression in general and, in particular, the new compression technique called fractal image compression. Fractal image compression is based on self-similarity, where one part of an image is similar to the other part of the same image. Low bit rate color image sequence coding is very important for video transmission and storage applications. The most significant aspect of this work is the development of color images using fractal-based color image compression, since little work has been done previously in this area. The results obtained show that the fractal-based compression works for the color images works as well as for the gray-scale images. Nevertheless, the encoding of the color images takes more time than the gray-scale images. Color images are usually compressed in a luminance-chrominance coordinate space, with the compression performed independently for each coordinate by applying the monochrome image processing techniques. For image sequence compression, the design of an accurate and efficient algorithm for computing motion to exploit the temporal redundancy has been one of the most active research areas in computer vision and image compression. Pixel-based motion estimation algorithms address pixel correspondence directly by identifying a set of local features and computing a match between these features across the frames. These direct techniques share the common pitfall of high computation complexity resulting from the dense vector fields produced. For block matching motion estimation algorithms, the quad-tree data structure is frequently used in image coding to recursively decompose an image plane into four non-overlapping rectangular blocks.

Hyperspectral Fluorescence Imaging for Mouse Skin Tumor Detection

This paper presents a hyperspectral imaging technique based on laser-induced fluorescence for non-invasive detection of tumorous tissue on mouse skin. Hyperspectral imaging sensors collect image data in a number of narrow, adjacent spectral bands. Such high-resolution measurement of spectral information reveals contiguous emission spectra at each image pixel useful for the characterization of constituent materials. The hyperspectral image data used in this study are fluorescence images of mouse skin consisting of 21 spectral bands in the visible spectrum of the wavelengths ranging from 440 nm to 640 nm. Fluorescence signal is measured with the use of laser excitation at 337 nm. An acousto-optic tunable filter (AOTF) is used to capture images at 10 nm intervals. All spectral band images are spatially registered with the reference band image at 490 nm to obtain exact pixel correspondences by compensating the spatial offsets caused by the refraction differences in AOTF at different wavelengths during the image capture procedure. The unique fluorescence spectral signatures demonstrate a good separation to differentiate malignant tumors from normal tissues for rapid detection of skin cancers without biopsy.

Correcting and matching time sequence images of plant leaves using Penalized Likelihood Warping and Robust Point Matching

Stress in plants can be measured using chlorophyll fluorescence imaging. The development of patterns in time can give an indication of the type of stress. Since leaves grow and show leaf movements, there is no pixel to pixel correspondence in time laps imaging data. In this article, Penalized Likelihood Warping and Robust Point Matching methods for recovering the pixel to pixel correspondence of a leaf within a time series are studied. It is shown that Robust Point Matching method is more suitable for our application than Penalized Likelihood Warping. Furthermore, Robust Point Matching method is much faster than Penalized Likelihood Warping. After warping an image sequence of a cabbage leaf infected with the bacteria Xanthomonas campestris pv. campestris, it was possible to identify infected spots 30 h after infection, where in unwarped images differences just can be seen 60 h after infection. Time series of the warped image data can be used to study and measure stress patterns in order to detect and identify diseases at an early stage.

Realization of high sensitivity displacement field from moiré interferometer with rough phase shifting mechanism and pattern matching technique

In this study, phase shifting method is used to modify moiré system into micro moiré interferometer which can measure displacement field with highly improved sensitivity. Apart from existing micro moiré technique, a low cost and less precise translation stage with rough resolution (10 μm resolution) is adapted for the phase shifter. Least square algorithm is applied to estimate the arbitrary phase shifted amount and to minimize the errors induced by lowering the cost. Moreover, specimen grating is phase shifted instead of reference grating which enables simple construction from given moiré system. To compensate for rigid body in-plane translation of specimen that may occur during phase shifting, pattern matching algorithm is put into practice to ensure pixel correspondence for each phase shifted images. To verify the newly constructed micro moiré technique, local displacement fields of Fine pitch Ball Grid Array package and Wafer Level Chip Size Package with elevated sensitivity up to 26 nm per fringe was acquired.

A similarity metric for edge images

The performance of several discrepancy measures for the comparison of edge images is analyzed and a novel similarity metric aimed at overcoming their problems is proposed. The algorithm finds an optimal matching of the pixels between the images and estimates the error produced by this matching. The resulting Pixel Correspondence Metric (PCM) can take into account edge strength as well as the displacement of edge pixel positions in the estimation of similarity. A series of experimental tests shows the new metric to be a robust and effective tool in the comparison of edge images when a small localization error of the detected edges is allowed.

Integrated surface model optimization for freehand three-dimensional echocardiography

The major obstacle of three-dimensional (3-D) echocardiography is that the ultrasound image quality is too low to reliably detect features locally. Almost all available surface-finding algorithms depend on decent quality boundaries to get satisfactory surface models. We formulate the surface model optimization problem in a Bayesian framework, such that the inference made about a surface model is based on the integration of both the low-level image evidence and the high-level prior shape knowledge through a pixel class prediction mechanism. We model the probability of pixel classes instead of making explicit decisions about them. Therefore, we avoid the unreliable edge detection or image segmentation problem and the pixel correspondence problem. An optimal surface model best explains the observed images such that the posterior probability of the surface model for the observed images is maximized. The pixel feature vector as the image evidence includes several parameters such as the smoothed grayscale value and the minimal second directional derivative. Statistically, we describe the feature vector by the pixel appearance probability model obtained by a nonparametric optimal quantization technique. Qualitatively, we display the imaging plane intersections of the optimized surface models together with those of the ground-truth surfaces reconstructed from manual delineations. Quantitatively, we measure the projection distance error between the optimized and the ground-truth surfaces. In our experiment, we use 20 studies to obtain the probability models offline. The prior shape knowledge is represented by a catalog of 86 left ventricle surface models. In another set of 25 test studies, the average epicardial and endocardial surface projection distance errors are 3.2 +/- 0.85 mm and 2.6 +/- 0.78 mm, respectively.

Stereo Matching Using Optic Flow

A method for the determination of pixel correspondence in stereo image pairs is presented. The optic flow vectors that result from the displacement of the point of projection are obtained and the correspondence between pixels of the various objects in the scene is derived from the optic flow vectors. The proposed algorithm is implemented and the correspondence vectors are obtained. Various specialized improvements of the method are implemented and thoroughly tested on sequences of image pairs giving rise to interesting conclusions. The algorithm is highly-parallelizable and therefore suitable for real-time applications.

ON THE RECOVERY OF MOTION AND STRUCTURE WHEN CAMERAS ARE NOT CALIBRATED

This paper addresses the problem of computing the camera motion and the Euclidean 3D structure of an observed scene using uncalibrated images. Given at least two images with pixel correspondences, the motion of the camera (translation and rotation) and the 3D structure of the scene are calculated simultaneously. We do not assume the knowledge of the intrinsic parameters of the camera. However, an approximation of these parameters is required. Such an approximation is all the time available, either from the camera manufacturer's data or from former experiments. Classical methods based on the essential matrix are highly sensitive to image noise. This sensitivity is amplified when the intrinsic parameters of the cameras contain errors. To overcome such instability, we propose here a method where a particular choice of a 3D Euclidean coordinate system with a different parameterization of the motion/structure problem allowed us to reduce significantly the total number of unknowns. In addition, the simultaneous calculation of the camera motion and the 3D structure has made the computation of the motion and structure less sensitive to the errors in the values of the intrinsic parameters of the camera. All steps of our method are linear. However, a final nonlinear optimal step might be added to improve the accuracy of the results and to allow the orthogonality of the rotation matrix to be taken into account. Experiments with real images validated our method and showed that a good quality motion/structure can be recovered from a pair of uncalibrated images. Intensive experiments with simulated images have shown the relationship between the errors on the intrinsic parameters and the accuracy of the recovered 3D structure.

Modelling vegetation patterns using natural and anthropogenic influence factors: preliminary experience with a GIS based model applied to an Alpine area

A modelling approach based on discriminant analysis and the GIS (geographical information system) is presented with the aim of identifying the influence of environmental parameters and human impact on the Alpine cultural landscape. For testing the model we used the vegetation patterns of an area of 5 km 2 between 1300 and 2300 m above sea level in the Central Alps (Passeier Valley, Italy). The overall prediction for the highly heterogeneous vegetation (a total of 21 different vegetation units characterized by high biodiversity) is in the order of 78% (pixel to pixel correspondence), the probability of correct classification by chance is only 21%. Only two vegetation units were totally misclassified, whereas six were classified correctly at almost 100%, a further seven at 60–80% and the remaining six at 20–50%. The proportion of unexplained variability in the research area may to some degree be attributed to the snow distribution pattern and to the use of a more detailed scale of spatial grazing habits. The most important lack of information, however, concerns land use history. Simulation analysis shows that human impact and elevation exert a major influence on the vegetation, whereas hydrological parameters and radiation do not greatly affect biodiversity and vegetation patterns in the study area.

Image Analysis for 3D Modeling, Rendering, and Virtual View Generation

The generation of virtual environments for telepresence systems, interactive viewing, and immersion in remote 3D-scenarios is an expanding and very promising research field. The challenge of this technology is to design systems capable of synthesizing views from any desired perspective using a set of real-scene perspectives. In such a system, two main parts can be distinguished: multiview image-analysis and viewpoint synthesis. Among other methods, multiview image-analysis is comprised of methods for the solution of three commonly encountered difficult tasks in computer vision: estimation of pixel correspondence, object segmentation, and camera calibration. Moreover, the results of multiview image-analysis are the required foundations for image synthesis. In this paper, we present a robust framework for stereo-image analysis, 3D modeling, and viewpoint synthesis. The image analysis part comprises disparity estimation by an improved block-matching algorithm and depth-based object segmentation using morphology. The basic idea of the system is to solve each of these tasks in direct interaction with the others. The results obtained in the partial solution of the correspondence problem are used to substantially simplify the segmentation task and vice versa. In the image-synthesis part, two tasks are considered: direct generation of arbitrary intermediate views, and extraction of the 3D structure of the scene using disparity results and camera parameters. The system has been successfully demonstrated in the context of virtual-view generation and object manipulation through 3D modeling.

Optical and electronic design of a calibrated multichannel electronic interferometer for quantitative flow visualization

Calibrated multichannel electronic interferometry is an electro-optic technique for performing phase shifting of transient phenomena. The design of an improved system for calibrated multichannel electronic interferometry is discussed. This includes a computational method for alignment of three phase-shifted interferograms and determination of the pixel correspondence. During calibration the phase, modulation, and bias of the optical system are determined. These data are stored electronically and used to compensate for errors associated with the path differences in the interferometer, the separation of the phase-shifted interferograms, and the measurement of the phase shift.

Occam algorithms for computing visual motion

The standard approach to computing motion relies on pixel correspondence. Computational schemes impose additional constraints, such as smoothness and continuity of the motion vector even though these are not directly related to pixel correspondence. This paper proposes an alternative to the multiple constraints approach. By drawing analogy with machine learning, motion is computed as a function that accurately predicts frames. The Occam-Razor principle suggests that among all functions that accurately predict the second frame from the first frame, the best predictor is the "simplest," and simplicity can be rigorously defined in terms of encoding length. An implementation of a practical algorithm is described. Experiments with real video sequences verify the algorithm assumptions by showing that motion in typical sequences can be accurately described in terms of a few parameters. Our particular choice of predictors produces results that compare very favorably with other image flow algorithms in terms of accuracy and compactness. It may, however, be too constrained to enable accurate recovery of 3D motion and structure.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

特集 マルチメディア時代の画像処理 フレーム間統合による高精細ディジタル画像の獲得

When an object in a scene is moving slowly and/or the camera is being moved, the observed image sequence contains rich information about both the moving object and the scene. In this case, it seems that we can enhance spatial resolution within a region of interest in the observed image frame by integrating pixels over multiple observed image frames. The interframe-integration task involves a method for estimating interframe pixel correspondence with real pixel accuracy. The principal existing block-matching algorithm can provide highly accurate correspondence estimates only in the case in which a rigid object undergoes translation in a direction parallel to the image plane, it cannot cope with more general types of motion such as rotation. Standard gradient-based algorithms cannot stably provide accurate correspondence estimates for real moving images. In this paper, we present a method for estimating interframe pixel correspondence with real pixel accuracy. The system uses iteratively warping quadrilateral patches covering the region of interest, and is an interframe integration method. Experimental simulations conducted on real moving image sequences demonstrate that the proposed method enhances spatial resolution considerably when a region of interest is limited to one planar surface of an object.