Real-time Segmentation Research Articles

Interaction features for prediction of perceptual segmentation: Effects of musicianship and experimental task

As music unfolds in time, structure is recognised and understood by listeners, regardless of their level of musical expertise. A number of studies have found spectral and tonal changes to quite successfully model boundaries between structural sections. However, the effects of musical expertise and experimental task on computational modelling of structure are not yet well understood. These issues need to be addressed to better understand how listeners perceive the structure of music and to improve automatic segmentation algorithms. In this study, computational prediction of segmentation by listeners was investigated for six musical stimuli via a real-time task and an annotation (non real-time) task. The proposed approach involved computation of novelty curve interaction features and a prediction model of perceptual segmentation boundary density. We found that, compared to non-musicians’, musicians’ segmentation yielded lower prediction rates, and involved more features for prediction, particularly more interaction features; also non-musicians required a larger time shift for optimal segmentation modelling. Prediction of the annotation task exhibited higher rates, and involved more musical features than for the real-time task; in addition, the real-time task required time shifting of the segmentation data for its optimal modelling. We also found that annotation task models that were weighted according to boundary strength ratings exhibited improvements in segmentation prediction rates and involved more interaction features. In sum, musical training and experimental task seem to have an impact on prediction rates and on musical features involved in novelty-based segmentation models. Musical training is associated with higher presence of schematic knowledge, attention to more dimensions of musical change and more levels of the structural hierarchy, and higher speed of musical structure processing. Real-time segmentation is linked with higher response delays, less levels of structural hierarchy attended and higher data noisiness than annotation segmentation. In addition, boundary strength weighting of density was associated with more emphasis given to stark musical changes and to clearer representation of a hierarchy involving high-dimensional musical changes.

Automatic Segmentation and Probe Guidance for Real-Time Assistance of Ultrasound-Guided Femoral Nerve Blocks

Ultrasound-guided regional anesthesia can be challenging, especially for inexperienced physicians. The goal of the proposed methods is to create a system that can assist a user in performing ultrasound-guided femoral nerve blocks. The system indicates in which direction the user should move the ultrasound probe to investigate the region of interest and to reach the target site for needle insertion. Additionally, the system provides automatic real-time segmentation of the femoral artery, the femoral nerve and the two layers fascia lata and fascia iliaca. This aids in interpretation of the 2-D ultrasound images and the surrounding anatomy in 3-D. The system was evaluated on 24 ultrasound acquisitions of both legs from six subjects. The estimated target site for needle insertion and the segmentations were compared with those of an expert anesthesiologist. Average target distance was 8.5 mm with a standard deviation of 2.5 mm. The mean absolute differences of the femoral nerve and the fascia segmentations were about 1–3 mm.

The future of remote ECG monitoring systems.

Remote ECG monitoring systems are becoming commonplace medical devices for remote heart monitoring. In recent years, remote ECG monitoring systems have been applied in the monitoring of various kinds of heart diseases, and the quality of the transmission and reception of the ECG signals during remote process kept advancing. However, there remains accompanying challenges. This report focuses on the three components of the remote ECG monitoring system: patient (the end user), the doctor workstation, and the remote server, reviewing and evaluating the imminent challenges on the wearable systems, packet loss in remote transmission, portable ECG monitoring system, patient ECG data collection system, and ECG signals transmission including real-time processing ST segment, R wave, RR interval and QRS wave, etc. This paper tries to clarify the future developmental strategies of the ECG remote monitoring, which can be helpful in guiding the research and development of remote ECG monitoring.

An exergame system based on force platforms and body key-point detection for balance training.

Postural instability affects a large number of people and can compromise even simple activities of the daily routine. Therapies for balance training can strongly benefit from auxiliary devices specially designed for this purpose. In this paper, we present a system for balance training that uses the metaphor of a game, what contributes to the motivation and engagement of the patients during a treatment. Such approach is usually named exergame, in which input devices for posturographic assessment and a visual output perform the interaction with the subject. The proposed system uses two force platforms, one positioned under the feet and the other under the hip of the subject. The force platforms employ regular load cells and a microcontroller-based signal acquisition module to capture and transmit the samples to a computer. Moreover, a computer vision module performs body key-point detection, based on real time segmentation of markers attached to the subject. For the validation of the system, we conducted experiments with 20 neurologically intact volunteers during two tests: comparison of the stabilometric parameters obtained from the system with those obtained from a commercial baropodometer and the practice of several exergames. Results show that the proposed system is completely functional and can be used as a versatile tool for balance training.

Real-time Segmentation of Yarn Images Based on an FCM Algorithm and Intensity Gradient Analysis

Real-time Segmentation of Yarn Images Based on an FCM Algorithm and Intensity Gradient Analysis

Multi-floor building heating models in MATLAB and Modelica environments

Buildings are one of the largest energy consumers in the world. In northern countries, buildings consume most of the energy for space heating owing to the predominating cold climate conditions. Today there is a trend to use Building Energy Management Systems (BEMS) in buildings to make the indoor environment more comfortable and to utilise energy in a more efficient way. Currently, BEMS lack a building heating model, and the control is often based on temperature zones. Integration of a good building heating model with BEMS may assist in monitoring the heating of buildings in an optimal way while saving energy. Hence, the goal is to develop a model that can be applied in on-line control with acceptable performance and accuracy. This article covers multi-floor physics-based building heating models developed in MATLAB and Modelica environments. The Modelica model uses the model components from Modelica Buildings Library and is more complex than the MATLAB model. The applicability of the two models in on-line control in BEMS principally depends on the accuracy of the predictions and prediction time. The prediction accuracy of both models is satisfactory while the Modelica model is robust. With the used computational power, the MATLAB model provides faster results compared to the Modelica model. More real-time experiments are needed for both models, and they can be applied in on-line control, depending on the model simplicity, available computational power and real-time segments in the system. In addition, the methodology used in the MATLAB model development is application independent and can be implemented in different natures of building configurations.

Automatic data-driven real-time segmentation and recognition of surgical workflow.

With the intention of extending the perception and action of surgical staff inside the operating room, the medical community has expressed a growing interest towards context-aware systems. Requiring an accurate identification of the surgical workflow, such systems make use of data from a diverse set of available sensors. In this paper, we propose a fully data-driven and real-time method for segmentation and recognition of surgical phases using a combination of video data and instrument usage signals, exploiting no prior knowledge. We also introduce new validation metrics for assessment of workflow detection. The segmentation and recognition are based on a four-stage process. Firstly, during the learning time, a Surgical Process Model is automatically constructed from data annotations to guide the following process. Secondly, data samples are described using a combination of low-level visual cues and instrument information. Then, in the third stage, these descriptions are employed to train a set of AdaBoost classifiers capable of distinguishing one surgical phase from others. Finally, AdaBoost responses are used as input to a Hidden semi-Markov Model in order to obtain a final decision. On the MICCAI EndoVis challenge laparoscopic dataset we achieved a precision and a recall of 91% in classification of 7 phases. Compared to the analysis based on one data type only, a combination of visual features and instrument signals allows better segmentation, reduction of the detection delay and discovery of the correct phase order.

이유자돈사에서 개별 돼지 모니터링을 위한 실시간 돼지 구분

To reduce huge losses in pig farms, weaning pigs with weak immune systems are required to be carefully supervised. Even if various researches have been performed for livestock monitoring environment, segmenting each pig from touching pigs is still entrenched as a difficult problem. In this paper, we propose a real-time segmentation method for moving pigs by using motion information in a 24-h video surveillance system. The experimental results with the videos obtained from a domestic pig farm illustrated the possibility for segmenting by using our proposed method in real-time.

Fast and accurate semiautomatic haptic segmentation of brain tumor in 3D MRI images

In this study, a novel virtual reality-based interactive method combined with the application of a graphical processing unit (GPU) is proposed for the semiautomatic segmentation of 3D magnetic resonance imaging (MRI) of the brain. The key point of our approach is to use haptic force feedback guidance for the selection of seed points in a bounded volume with similar intensity and gradient. For the automatic determination of a bounded volume of segmentation in real time, parallel computation on the GPU is used. Automatic segmentation is applied in this adjustable bounded spherical volume with a variable diameter, which is controlled according to the edge map acquired from the gradient map. The haptic force feedback is used in order to guide the user to remain in a volume, where the intensity and gradient change are under a defined threshold range. After each seed point selection, the segmentation algorithm works inside the bounded volume of the ball with an adjusted diameter. The proposed segmentation method based on force and visual feedback with the advantage of adjustable bounded volume is not only accurate and effective in narrow spaces near the boundaries of different layers, but also fast in large homogeneous spaces since the radius of the ball increases in such regions. Parallel programming on the GPU is used for computing gradient change in selected directions, which is needed for the self-adjustment of the sphere diameter. Gradient values are used for calculating the haptic force on the CPU in real time. In this study, two haptic devices are used, one for getting haptic force feedback and the other for camera guidance during 3D visualization. A comparison between manual segmentation of MRI by an expert surgeon and the proposed segmentation algorithm is done. The proposed segmentation procedure is completed 4 times faster than the manual segmentation with similar accuracy.

A collaborative client participant fusion system for realistic remote conferences

Remote conferencing systems provide a shared environment where people in different locations can communicate and collaborate in real time. Currently, remote video conferencing systems present separate video images of the individual participants. To achieve a more realistic conference experience, we enhance video conferencing by integrating the remote images into a shared virtual environment. This paper proposes a collaborative client participant fusion system using a real-time foreground segmentation method. In each client system, the foreground pixels are extracted from the participant images using a feedback background modeling method. Because the segmentation results often contain noise and holes caused by adverse environmental lighting conditions and substandard camera resolution, a Markov Random Field model is applied in the morphological operations of dilation and erosion. This foreground segmentation refining process is implemented using graphics processing unit programming, to facilitate real-time image processing. Subsequently, segmented foreground pixels are transmitted to a server, which fuses the remote images of the participants into a shared virtual environment. The fused conference scene is represented by a realistic holographic projection.

Topology‐aware moving least square deformation for 2D characters

AbstractDeformation method based on moving least squares (MLS) allows the user to manipulate 2D characters using either sets of points or line segments in real time. However, the traditional MLS deformation spreads the deformation of the controls with respect to the spatial distance, but oblivious to the shape topology, which would possibly lead to distortion. In this paper, we present a topology‐aware MLS deformation approach for 2D characters. First, a Laplace equation is solved to obtain a set of weights, which are called harmonic weights. Then, the MLS deformation is performed by using the harmonic weights as the deformation influence of the user‐specified controls. Finally, the possible distortion in the traditional MLS deformation can be effectively avoided, as the harmonic weights spread the deformation of the controls in a localized and topology‐aware way. In addition, a simple but effective area‐preserving variant of MLS deformation is proposed, which is suitable for the editing of incompressible objects. Copyright © 2015 John Wiley & Sons, Ltd.

Real-time traffic incident detection using a probabilistic topic model

Traffic congestion occurs frequently in urban settings, and is not always caused by traffic incidents. In this paper, we propose a simple method for detecting traffic incidents from probe-car data by identifying unusual events that distinguish incidents from spontaneous congestion. First, we introduce a traffic state model based on a probabilistic topic model to describe the traffic states for a variety of roads. Formulas for estimating the model parameters are derived, so that the model of usual traffic can be learned using an expectation–maximization algorithm. Next, we propose several divergence functions to evaluate differences between the current and usual traffic states and streaming algorithms that detect high-divergence segments in real time. We conducted an experiment with data collected for the entire Shuto Expressway system in Tokyo during 2010 and 2011. The results showed that our method discriminates successfully between anomalous car trajectories and the more usual, slowly moving traffic patterns.

Locally rigid, vessel-based registration for laparoscopic liver surgery.

PurposeLaparoscopic liver resection has significant advantages over open surgery due to less patient trauma and faster recovery times, yet is difficult for most lesions due to the restricted field of view and lack of haptic feedback. Image guidance provides a potential solution but is challenging in a soft deforming organ such as the liver. In this paper, we therefore propose a laparoscopic ultrasound (LUS) image guidance system and study the feasibility of a locally rigid registration for laparoscopic liver surgery.MethodsWe developed a real-time segmentation method to extract vessel centre points from calibrated, freehand, electromagnetically tracked, 2D LUS images. Using landmark-based initial registration and an optional iterative closest point (ICP) point-to-line registration, a vessel centre-line model extracted from preoperative computed tomography (CT) is registered to the ultrasound data during surgery.ResultsUsing the locally rigid ICP method, the RMS residual error when registering to a phantom was 0.7 mm, and the mean target registration error (TRE) for two in vivo porcine studies was 3.58 and 2.99 mm, respectively. Using the locally rigid landmark-based registration method gave a mean TRE of 4.23 mm using vessel centre lines derived from CT scans taken with pneumoperitoneum and 6.57 mm without pneumoperitoneum.ConclusionIn this paper we propose a practical image-guided surgery system based on locally rigid registration of a CT-derived model to vascular structures located with LUS. In a physical phantom and during porcine laparoscopic liver resection, we demonstrate accuracy of target location commensurate with surgical requirements. We conclude that locally rigid registration could be sufficient for practically useful image guidance in the near future.

Using Algorithms on Smart Transducer: An IEEE Standard Perspective

This paper describes the role of algorithm development in the IEEE standard 21451.001. The purpose is to promote and enhance the computational capabilities of smart transducers and facilitate the flow from sensor raw data processing to sensor knowledge fusion. A case study on real-time segmentation and labeling is conducted to demonstrate the feasibility of the first layer algorithm. The key role of the IEEE 21451.001 standard is to standardize the data structure for user-defined application code and pattern learning algorithms and regulate the protocols for data sharing and exchange. The considerations for the standard development are illustrated and discussed. The foreseeable impact of the standard for algorithms is highlighted as well.

A new intelligent method for minerals segmentation in thin sections based on a novel incremental color clustering

Mineral segmentation in thin sections is a challenging, popular, and important research topic in computational geology, mineralogy, and mining engineering. Mineral segmentation in thin sections containing altered minerals, in which there are no evident and close boundaries, is a rather complex process. Most of the thin sections created in industries include altered minerals. However, intelligent mineral segmentation in thin sections containing altered minerals has not been widely investigated in the literature, and the current state of the art algorithms are not able to accurately segment minerals in such thin sections. In this paper, a novel method based on incremental learning for clustering pixels is proposed in order to segment index minerals in both thin sections with and without altered minerals. Our algorithm uses 12 color features that are extracted from thin section images. These features include red, green, blue, hue, saturation and intensity, under plane and cross polarized lights in maximum intensity situation. The proposed method has been tested on 155 igneous samples and the overall accuracy of 92.15% and 85.24% has been obtained for thin sections without altered minerals and thin sections containing altered minerals, respectively. Experimental results indicate that the proposed method outperforms the results of other similar methods in the literature, especially for segmenting thin sections containing altered minerals. The proposed algorithm could be applied in applications which require a real time segmentation or efficient identification map such as petroleum geology, petrography and NASA Mars explorations.

Real-time segmentation methods for monocular soccer videos

The analysis of video records of soccer games is a topic of extensive and challenging research in image sequence processing, particularly that of player and field tracking. In this paper, we propose a new approach for detecting players and field lines in monocular TV video data that involves determining the convex field area and grass colors. This is carried out by considering contextual knowledge, together with a new method for color segmentation that selects polyhedrons in a frame-wise manner within the RGB cube. In summary, for every input image, a binary mask of the field area and a background mask are determined. Both mask creation algorithms achieve per pixel F1 scores of about 98% within our representative test set and are real-time capable. Applications like line detection and player tracking are presented.

FPGA Implementation for Real Time Video Segmentation Using Gausian Mixture Model

Real time segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated visual surveillance human-machine interface and very low bandwidth telecommunications. Background identification is a common feature in many video processing systems. One of the most important back ground identification algorithm is the Gaussian Mixture Model algorithm (GMM).On implementation of the Gaussian mixture model on results in Reduction of the processing capability of the overall system. Trainable Segmentation is adapted to improve the processing capability. After analyzing and evaluating the performance we conclude with several promising directions for future research.

A parallel LEGION algorithm and cell-based architecture for real time split and merge video segmentation

Split and merge segmentation is a popular region-based segmentation scheme for its robustness and computational efficiency. But it is hard to realize for larger size images or video frames in real time due to its iterative sequential data flow pattern. A quad-tree data structure is quite popular for software implementation of the algorithm, where a local parallelism is difficult to establish due to inherent data dependency between processes. In this paper, we have proposed a parallel algorithm of splitting and merging which depends only on local operations. The algorithm is mapped onto a hierarchical cell network, which is a parallel version of Locally Excitory Globally Inhibitory Oscillatory Network (LEGION). Simulation results show that the proposed design is faster than any of the standard split and merge algorithmic implementations, without compromising segmentation quality. The timing performance enhancement is manifested in its Finite State Machine based VLSI implementation in VIRTEX series FPGA platforms. We have also shown that, though segmentation qualitywise split-and-merge algorithm is little bit behind the state-of-the-art algorithms, computational speedwise it over performs those sophisticated and complex algorithms. Good segmentation performance with minimal computational cost enables the proposed design to tackle real time segmentation problem in live video streams. In this paper, we have demonstrated live PAL video segmentation using VIRTEX 5 series FPGA. Moreover, we have extended our design to HD resolution for which the time taken is less than 5 ms rendering a processing throughput of 200 frames per second.

Next Generation Network Real-Time Kinematic Interpolation Segment to Improve the User Accuracy

This paper demonstrates that automatic selection of the right interpolation/smoothing method in a GNSS-based network real-time kinematic (NRTK) interpolation segment can improve the accuracy of the rover position estimates and also the processing time in the NRTK processing center. The methods discussed and investigated are inverse distance weighting (IDW); bilinear and bicubic spline interpolation; kriging interpolation; thin-plate splines; and numerical approximation methods for spatial processes. The methods are implemented and tested using GNSS data from reference stations in the Norwegian network RTK service called CPOS. Data sets with an average baseline between reference stations of 60–70 km were selected. 12 prediction locations were used to analyze the performance of the interpolation methods by computing and comparing different measures of the goodness of fit such as the root mean square error (RMSE), mean square error, and mean absolute error, and also the computation time was compared. Results of the tests show that ordinary kriging with the Matérn covariance function clearly provides the best results. The thin-plate spline provides the second best results of the methods selected and with the test data used.

Integral split-and-merge methodology for real-time image segmentation

The segmentation of images is a critical step in many computer vision applications. Additionally, some applications require the achievement of acceptable segmentation quality while the algorithm is executed in real time. In this study, we present a split-and-merge segmentation methodology that uses integral images to improve the execution time. We call our methodology integral split and merge (ISM) segmentation. The integral images are used here to calculate statistics of the image regions in constant time. Those statistics are used to guide the splitting process by identifying the homogeneous regions in the image. We also propose a merge criterion that performs connected component analysis of the homogeneous regions. Moreover, the merging procedure is able to group regions of the image showing gradients. Furthermore, the number of regions resulting from the seg- mentation process is determined automatically. In a series of tests, we compare ISM against other state-of-the- art algorithms. The results from the tests show that our ISM methodology obtains image segmentations with a comparable quality, using a simple texture descriptor instead of a combination of color-texture descriptors. The proposed ISM methodology also has a piecewise linear computational complexity, resulting in an algorithm fast enough to be executed in real time. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, includ- ing its DOI. (DOI: 10.1117/1.JEI.24.1.013007)