Real-time Video Sequences Research Articles

Perioperative characterization of anastomotic doughnuts with high-resolution probe-based confocal laser endomicroscopy in colorectal cancer surgery: a feasibility study

Confocal laser enables in vivo real-time histopathology of the mucosa layer of gastrointestinal tract. The aim of this study was to assess the feasibility and the role of probe-based confocal laser endomicroscopy in extemporary examination of staple rings of patients with colorectal cancer. This was a prospective, single-center pilot study. Patients who underwent end-to-end stapled surgical resection for colorectal cancer were included. Confocal imaging was analyzed with great attention to image quality evaluation of cellular morphology and cellular structures of the serosal, muscular, and mucosal layers of the doughnuts than comparing results with definitive histopathology. Twenty-six doughnuts were analyzed. Real-time video sequences were obtained in all patients, with a total of 204 mosaic images. For each doughnut, most of the images were adequate for evaluation of cellular morphology and cellular structure of the serosal, muscular, and mucosal layers. Perioperative assessment of doughnut tissues in patients with colorectal cancer by confocal laser endomicroscopy is feasible and safe. Prospective studies are warranted for further evaluation of the clinical impact of this technology during surgery.

Cube2Video: Navigate Between Cubic Panoramas in Real-Time

Online virtual navigation systems enable users to hop from one 360° panorama to another, which belong to a sparse point-to-point collection, resulting in a less pleasant viewing experience. In this paper, we present a novel method, namely Cube2Video, to support navigating between cubic panoramas in a video-viewing mode. Our method circumvents the intrinsic challenge of cubic panoramas, i.e., the discontinuities between cube faces, in an efficient way. The proposed method extends the matching-triangulation-interpolation procedure with special considerations of the spherical domain. A triangle-to-triangle homography-based warping is developed to achieve physically plausible and visually pleasant interpolation results. The temporal smoothness of the synthesized video sequence is improved by means of a compensation transformation. As experimental results demonstrate, our method can synthesize pleasant video sequences in real time, thus mimicking walking or driving navigation.

A Method for Real-Time Medaka (<i>Oryzias latipes</i>) Detection Base on Machine Vision Technology

The Japanese medaka (Oryzias latipes) behavior monitoring system based on computer vision is very effective in the field of monitoring the safety of drinking water. But it is very difficult to detect the fishs body accurately and rapidly. In order to get the whole data of medaka in the real-time video sequence, a new approach to detect medaka is proposed in this paper which is based on 2D auto-threshold (OTSU), frame difference and background subtraction. With this method, the pixels in the image are divided into four parts: foreground, ghosting, foreground aperture and the others, then update background base on the data of the third part. At last, the whole fishs body can be got by some postprocessor. These methods are proved to be efficient to get the clean foreground and detect the fish in the real-time fish monitoring video.

Development of a SoC for Digital Television Set-Top Box: Architecture and System Integration Issues

This work presents the integration of several IPs to generate a system-on-chip (SoC) for digital television set-top box compliant to the SBTVD standard. Embedded consumer electronics for multimedia applications like video processing systems require large storage capacity and high bandwidth memory. Also, those systems are built from heterogeneous processing units, designed to perform specific tasks in order to maximize the overall system efficiency. A single off-chip memory is generally shared between the processing units to reduce power and save costs. The external memory access is one bottleneck when decoding high-definition video sequences in real time. In this work, a four-level memory hierarchy was designed to manage the decoded video in macroblock granularity with low latency. The use of the memory hierarchy in the system design is challenging because it impacts the system integration process and IP reuse in a collaborative design team. Practical strategies used to solve integration problems are discussed in this text. The SoC architecture was validated and is being progressively prototyped using a Xilinx Virtex-5 FPGA board.

Analysis and VLSI Implementation of EWA Rendering for Real-Time HD Video Applications

Nonlinear image warping or image resampling is a necessary step in many current and upcoming video applications, such as video retargeting, stereoscopic 3-D mapping, and multiview synthesis. The challenges for real-time resampling include not only image quality but also available energy and computational power of the employed device. In this paper, we employ an elliptical-weighted average (EWA) rendering approach to 2-D image resampling. We extend the classical EWA framework for increased visual quality and provide a very large scale integration architecture for efficient view rendering. The resulting architecture is able to render high-quality video sequences in real time targeted for low-power applications in end-user display devices.

An Efficient Approach for Real Time Tracking of Intruder and Abandoned Object in Video Surveillance System

Object Detection and Intruder detection is one of the important tasks in video surveillance system. This paper proposes an integrated approach for the tracking of abandoned and unknown objects using background subtraction and morphological filtering. The aim of the approach is to automatically recognize activities around restricted area to improve safety and security of the servicing area by multiplexing hundreds of video streams in real time. The tracking module takes as input per camera tracking and recognition results and fuses these into object estimation. A novel algorithm for object tracking in video pictures, based on image segmentation is proposed. With the image segmentation all objects in images can be detected whether they are moving or not by using image segmentation results of successive frames. Consequently, the proposed algorithm can be applied to multiple movements. The algorithm was tested on real time video surveillance system and it produces very low false alarms and missing detection. This approach definitely provides security and detects the moving object in real time video sequence and live video streaming.

Quality time-of-flight range imaging for feature-based registration using bacterial foraging

Image registration is a widely tackled research topic in the computer vision and the computer graphics fields. This problem aims to find an optimal transformation or correspondence between images acquired under different conditions. Recently, a new 3D image acquisition device based on the time-of-flight technology has appeared which obtains range images from real-time 3D video sequences. In this contribution, we aim to study the feasibility of using this new class of cameras to face the 3D model reconstruction procedure. Our proposal is two-fold. First, we introduce a novel image preprocessing pipeline in order to improve the quality of time-of-flight range images and a subsequent feature extraction method considering both 2D and 3D images. As second major objective, we propose an adaptation of the evolutionary bacterial foraging optimization algorithm, which has recently emerged as a very powerful technique for real parameter optimization and gained a high interest for distributed optimization and control, to tackle the range image registration problem. Finally, we analyse the performance of our proposal against other state-of-the-art evolutionary image registration methods.

Abstract 4063: Image-guided fluorescent endomicroscopy of glioblastoma with SVV-GFP viral labeling

Abstract We propose a novel method to improve detection of infiltrative microscopic GBM tumors during surgery, by labeling the infiltrative tumor cells in vivo by a single intravenous injection of a highly specific tumor seeking fluorescence virus, Seneca Valley Virus (SVV-GFP). The feasibility of this approach was demonstrated in a permissive pediatric GBM (IC-2305GBM) and an adult GBM (IC-A46GBM) orthotopic xenograft mouse model. Both models were derived from primary tissue of GBM from patients to preservethe histopathological and immunophenotypical characteristics of the original tumors. After establishing the xenograft tumors, SVV-GFP was injected intravenously at a dose of 5x1014 viral particles/kg. The animals were imaged at day 3 after SVV-GFP virus injection. During the imaging procedure, the animals were euthanized through anesthesia overdose and a small window was immediately drilled in the skull to introduce a 1mm outer diameter fiberoptic fluorescence probe through the tumor. Fluorescent labeled cells were recorded as a real-time video sequence along the track of the probe. Image quantification starts with low frequency temporal drift correction using Fast Fourier Transform based lowpass filtering. Air image frames (before probe insertion) were used as a zero reference and the intensity range of the video is shifted accordingly. A normal tissue fluorescence image was then estimated using a median statistics of all image frames acquired in normal brain tissue. This image was used to correct for lower detection sensitivity near the probe edge or any field inhomogeneity. The video sequence was then normalized to the normal tissue image to create a TBR map. By definition, air is used as reference with TBR=0 and normal tissue is at TBR=1. A cutoff of TBR>1.1 is used and tumor cells exceeding that threshold were classified as GFP-positive tumor cells. Fractional area occupied by the virus-labeled microscopic tumors were computed with the TBR exceeding 1.1 within the field of view. A low fractional area (<0.01%) was found in normal brain tissue using this method, demonstrating that our method is robust. To summarize, we demonstrated an innovative approach for microscopic tumor detection in glioblastoma that is two to three orders of magnitude smaller than those currently detected with in vivo imaging modalities.This method can potentially be applied to detect microscopic tumors in metastatic retinoblastoma, medulloblastoma, endocrine tumors and small cell lung cancer which are shown to be permissive to infection of this virus. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 4063. doi:1538-7445.AM2012-4063

Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors

This paper proposes an algorithm for feedforward categorization of objects and, in particular, human postures in real-time video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified line segment Hausdorff distance classifier combined with on-the-fly cluster-based size and position invariant categorization. The system can achieve about 90 percent average success rate in the categorization of human postures, while using only a small number of training samples. Compared to state-of-the-art bio-inspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least five times, and is an ideal candidate for hardware implementation with event-based circuits.

LoPP: Locality Preserving Projections for Moving Object Detection

Automatic moving object detection and tracking is very important task in video surveillance applications. In this paper, we propose a novel scheme for moving object detection based on Locality Preserving Projections (LPP). It is also known as Laplacian eigenmaps, which optimally preserves the neighborhood structure of the data set [1]. The proposed method was tested on standard PETS dataset and many real time video sequence and the results was satisfactory.

In Situ X-Ray Video Microscopy as a Tool in Solidification Science

Over the last 20 years, time-resolved high-resolution x-ray imaging techniques have become established as methods of choice for in situ studies of solidification microstructure formation in metallic alloys. Using high-brilliance x-radiation delivered at modern synchrotrons, combined with the most recent advances in x-ray cameras and optics, it is now possible to capture real-time video sequences with projection image resolutions down to 100 nm at frame rates of 20 Hz. This article presents some illustrative examples of the current capabilities of in situ x-ray video microscopy in terms of radiographic and topographic imaging as a platform for studying microstructure pattern formation, growth dynamics, and other topics related to metallic alloy solidification.

3D information extraction using Region-based Deformable Net for monocular robot navigation

This paper extends the Region-based Deformable Net (RbDN) technique described in [1] to extract the 3D information of all the objects in the scene from a single moving camera. The technique is used for segmenting real-time video sequences captured from a single moving camera. The deformation process tracks the changes in the location and the shape of the segments across the frames. These changes along with the camera displacement are used to estimate the 3D information. The algorithm is completely autonomous and does not require pre-knowledge, training, or assumption about the contents of the sequence. It can handle the difficult case where the motion of the camera is parallel to its optical axis. It can also estimate the distances to objects that are more than 100 m away as long as the camera displacement is over 10% of the expected distance to the objects.

Robust Range Finder Through a Laser Pointer and a Webcam

Currently, there is a great interest in the study of three-dimensional reconstruction from digital images. The applications of photogrammetric algorithms has allowed us significant improvements in procedures of camera calibration, movement of objects in scenes, shape from shading and range images. The procedure of the distance estimation from simple images is important to find the depth measurements in any procedure of three-dimensional reconstruction of scenes. In this work, we describe an adaptation of a scanner prototype based on a laser pointer and a webcam. It was applied to the robust estimation of absolute distance on images obtained from real time video sequences. Experimental tests were performed in order to demonstrate the effectiveness of the distance calculation in real time through a geometric model and a simple system of linear regression. From a wide data set of tests with different scanning parameter, good results on range finding were obtained.

Security System and Surveillance Using Real Time Object Tracking and Multiple Cameras

In this paper we propose multiple cameras using real time tracking for surveillance and security system. It is extensively used in the research field of computer vision applications, like that video surveillance, authentication systems, robotics, pre-stage of MPEG4 image compression and user inter faces by gestures. The key components of tracking for surveillance system are extracting the feature, background subtraction and identification of extracted object. Video surveillance, object detection and tracking have drawn a successful increased interest in recent years. A object tracking can be understood as the problem of finding the path (i.e. trajectory) and it can be defined as a procedure to identify the different positions of the object in each frame of a video. Based on the previous work on single detection using single stationary camera, we extend the concept to enable the tracking of multiple object detection under multiple camera and also maintain a security based system by multiple camera to track person in indoor environment, to identify by my proposal system which consist of multiple camera to monitor a person. Present study mainly aims to provide security and detect the moving object in real time video sequences and live video streaming. Based on a robust algorithm for human body detection and tracking in videos created with support of multiple cameras.

A Lossless Embedded Compression Using Significant Bit Truncation for HD Video Coding

Increasing the image size of a video sequence aggravates the memory bandwidth problem of a video coding system. Despite many embedded compression (EC) algorithms proposed to overcome this problem, no lossless EC algorithm able to handle high-definition (HD) size video sequences has been proposed thus far. In this paper, a lossless EC algorithm for HD video sequences and related hardware architecture is proposed. The proposed algorithm consists of two steps. The first is a hierarchical prediction method based on pixel averaging and copying. The second step involves significant bit truncation (SBT) which encodes prediction errors in a group with the same number of bits so that the multiple prediction errors are decoded in a clock cycle. The theoretical lower bound of the compression ratio of the SBT coding was also derived. Experimental results have shown a 60% reduction of memory bandwidth on average. Hardware implementation results have shown that a throughput of 14.2 pixels/cycle can be achieved with 36 K gates, which is sufficient to handle HD-size video sequences in real time.

2.02] Confocal endomicroscopy (CFE) in gastroenterology: Ready for prime time?

Objective Confocal laser endomicroscopy (CFE), when implemented during endoscopy, enables in vivo microscopy of the mucosal layer of any tissue in the gastrointestinal tract with subcellular resolution. Endomicroscopy opens the door to immediate tissue and vessel analysis. Different types of diseases can be diagnosed with optical surface and subsurface analysis. Discrimination between normal and malignant cells is possible. Analysis of the in vivo microarchitecture can be used for targeting biopsies to relevant areas, and subsurface imaging can unmask microscopic diseases. The aim of our study was to validate this new technique further. Material and methods For CFE we used a Cellvizio® miniprobe, which is compatible for all endoscopes with an operating channel of 2.6 mm. Placed in contact with the mucosa, the miniprobe delivers real-time video sequences of the tissue microarchitecture at 12 images per second and a depth resolution of about 50 μm. We used CFE to perform “optical biopsies” in patients with known Barrett's esophagus, one with a known high-grade intraepithelial neoplasia, in the follow-up of a patient after small bowel transplantation, and in a patient with invasive cholangiocellular carcinoma of the bile duct. Additionally, we performed CFE in a newly established pig model of benign bile duct stenosis. The benign stenosis were induced by endoscopic retrograde cholangiography (ERC) with introduction of a 10 Fr-heater probe and application of HF energy (400 J) approximately 3 cm proximal of the papillae. Results Confocal imaging of the normal squamous epithelium of the esophagus demonstrated squamous cells at high resolution, showing capillaries (filled with red blood cells) within single papillae. The typical shape of Barrett's epithelium was villous, differing from the cardiac epithelium. High-grade intraepithelial neoplasias can be recognized by a distinct cell type in endomicroscopy. Nevertheless, this technique appears to require experience and is not an “easy-to-use-method” in the esophagus. In contrast, CFE appears to be safe and useful for monitoring patients for transplant rejection after small bowel transplantation. In bile duct cancer, CFE showed a deranged vascular architecture in contrast to the regular pattern obtained within the normal epithelium of this patient. In our pig model we were able to establish a reproducible benign bile duct stenosis. These fibrotic structures were characterized by virtually no CFE signal. This is in direct contrast to the normal bile duct epithelium of animals and to the bile duct cancer mentioned above. Conclusion CFE is feasible in a wide variety of gastrointestinal tissues using regular endoscopes and a thin miniprobe. Although real-time “optical biopsies” are achievable, the exact diagnosis still has to be made by the pathologist. CFE is helpful for discrimination between normal tissue and dysplasia or even carcinoma. The application in the bile duct system probably requires an additional cholangioscope for placing the miniprobe specifically in the suspected area. Further studies are necessary to evaluate this promising technique in bile-duct disorders in humans.

Anomaly Detection Based on Motion Direction

A novel algorithm is proposed in this paper to detect anomalous human behaviors based on motion directions. According to different motion direction rules for different events, we introduce block-based motion directions to model those events, and use support vector machine (SVM) to detect the abnormalous actions from real-time monitoring video sequences. To increase the robustness against noise and to capture the slight movement of the object, we select the foreground frames (the frames having human object) with a background edge model before the action feature extraction. Then, action features are extracted using normalized histogram analysis from the motion directions of all the foreground frames. Our experiments on public areas such as hallway show that our algorithm is able to track complex actions of single and multiple people accurately and is robust against the variation of object size, lighting, and noise during their movements. Our algorithm is of low computation complexity thus it can be used for real time monitoring.

In Vivo Histopathology with Portable Confocal Miniprobes Endomicroscopy in the Diagnosis of Celiac Disease (CD)

Background: Endoscopic duodenal biopsy is still critical in the diagnosis of CD, but often sampling errors and poor quality specimens may generate false-negative results. Confocal endomicroscopy (CEM) allows an in vivo histologic assessment of the mucosal layer during routine endoscopy. Aim: To evaluate the application of CEM in the diagnosis of CD and the potential benefit in the management of patients. Methods: CEM (Mauna Kea Tech., Paris, France), using portable confocal mini probes (Type Z, Type MiniO) was performed in 12 patients (8 F/4 M; mean age: 31 yrs; range 19-44 yrs) underwent an upper GI endoscopy for either suspected CD or malabsorption symptoms. Endoscopy was performed under conscious sedation (i.v. meperidine and midazolam) with standard video gastroscopes. We used endovenous fluorescein dye (2,5-5 mL), to enhance the diagnostic power of CEM. Real time video sequences were digitally recorded. Multiple biopsies were taken from the same examined area. All stored sequences were put into a random order and assessed by a gastroenterologist blinded to any histopathological data. Biopsies were analized by experts pathologists blinded to CEM data. Traditional histology was considered as the gold standard. Villous atrophy (VA) (presence of 5 or fewerblunt-shaped villi seen on superficial scans) and crypt hypertrophy (CH) (>1 crypt on imaging of the deep mucosa) were defined as CEM features of CD. Results: The patient's clinical-pathological characteristics are shown in the table. Comparing the CEM data with histology, we can assess a sensibility of 90%, a specificity of 100%, and an accuracy of 92%. Conclusions: This is our preliminary experience with CEM applied to the diagnosis of CD. It allows a real time in vivo histological improving histology itself, reducing the number of biopsy, guiding a better tissue sampling.

Fiberoptic Imaging of Cavernous Nerves In Vivo

A critical intraoperative variable for the return of tumescence following radical prostatectomy is preservation of the cavernous nerves. We developed a nontoxic technique that would allow high resolution, in vivo real-time imaging specifically of the cavernous nerves. The cavernous nerves were labeled by injecting a fluorescent retrograde nerve tracer into the corpus cavernosum of male rats. Nerves were subsequently imaged in vivo using fiberoptic confocal fluorescent microscopy. Initial screening trials were performed to decide on a nerve tracer capable of axonal labeling, optimize injection concentration and characterize retrograde transport time. Toxicity studies included intracavernous pressure monitoring following electrical nerve stimulation, apoptotic staining of injected cavernous tissue and measurement of lipid peroxidation in nerves exposed to laser emissions during imaging. In vivo real-time video sequences of fluorescently labeled cavernous nerves were recorded. The screening trial indicated that the B subunit of cholera toxin conjugated to AlexaFluor 488 (Invitrogen) provided optimal imaging after 9 days of retrograde transport. Toxicity studies showed that maximal intracavernous pressure responses did not differ between labeled and unlabeled nerves (p = 0.9671). Tracer injection did not increase apoptosis in cavernous tissue and laser exposure did not increase lipid peroxidation in nerves. In vivo real-time imaging of the cavernous nerves is possible with no measurable toxicity, allowing the maintenance of erection. This novel imaging modality may allow urologists to identify cavernous nerves during pelvic surgery.

Hardware implementation of a disparity estimation scheme for real-time compression in 3D imaging applications

This paper presents a novel hardware implementation of a disparity estimation scheme targeted to real-time Integral Photography (IP) image and video sequence compression. The software developed for IP image compression achieves high quality ratios over classic methodologies by exploiting the inherent redundancy that is present in IP images. However, there are certain time constraints to the software approach that must be confronted in order to address real-time applications. Our main effort is to achieve real-time performance by implementing in hardware the most time-consuming parts of the compression algorithm. The proposed novel digital architecture features minimized memory read operations and extensive simultaneous processing, while taking into concern the memory and data bandwidth limitations of a single FPGA implementation. Our results demonstrate that the implemented hardware system can successfully process high resolution IP video sequences in real-time, addressing a vast range of applications, from mobile systems to demanding desktop displays.