Computer Vision-based System Research Articles

An automatic system for the detection of dairy cows lying behaviour in free-stall barns

In this paper, a method for the automatic detection of dairy cow lying behaviour in free-stall barns is proposed. A computer visionbased system (CVBS) composed of a video-recording system and a cow lying behaviour detector based on the Viola Jones algorithm was developed. The CVBS performance was tested in a head-to-head free stall barn. Two classifiers were implemented in the software component of the CVBS to obtain the cow lying behaviour detector. The CVBS was validated by comparing its detection results with those generated from visual recognition. This comparison allowed the following accuracy indices to be calculated: the branching factor (BF), the miss factor (MF), the sensitivity, and the quality percentage (QP). The MF value of approximately 0.09 showed that the CVBS missed one cow every 11 well detected cows. Conversely, the BF value of approximately 0.08 indicated that one false positive was detected every 13 well detected cows. The high value of approximately 0.92 obtained for the sensitivity index and that obtained for QP of about 0.85 revealed the ability of the proposed system to detect cows lying in the stalls.

A computer vision-based system for the automatic detection of lying behaviour of dairy cows in free-stall barns

A computer vision-based system for the automatic detection of dairy cow lying behaviour in free-stall barns is proposed. The system is composed of a multi-camera video-recording system and a software component which executes a cow lying behaviour detector model using the Viola–Jones algorithm. A method to carry out the training, testing and validation phase of the modelled cow lying behaviour detector is described. The performance of the system was tested in an area of a head-to-head free-stall barn where a group of 15 Holstein dairy cows was housed. A multi-camera video-recording system was installed to obtain panoramic top-view images of the area under study. Since the Viola–Jones algorithm was not invariant to the rotation of the cow images, two classifiers were modelled, one for each row of stalls located in the barn. These two classifiers were implemented in the software component of the system in order to perform the lying behaviour detection. The system was validated by comparing its detection results with those generated from visual recognition. The ability of the system to detect cow lying behaviour was confirmed by the high value of its sensitivity, which was approximately 92%. Conversely, the value of the branching factor which was approximately 0.08 indicated that one false positive was detected for every 13 well detected cows. These results suggest that the system proposed in this study could be used for the calculation of the cow lying index which is widely used to investigate cow lying behaviour in free-stall barns. ► To model the cow lying behaviour detector image enhancement was not required. ► In the training phase FPR cas of the order of 10 −7 and TPR cas of 0.90 were achieved. ► The testing phase proved detector ability even when noise affected camera images. ► The cow lying behaviour detector provided a sensitivity of 92% and a BF of 0.08. ► CVBS can be used to compute cow lying index.

A Posture Recognition-Based Fall Detection System for Monitoring an Elderly Person in a Smart Home Environment

We propose a novel computer vision based fall detection system for monitoring an elderly person in a home care application. Background subtraction is applied to extract the foreground human body and the result is improved by using certain post-processing. Information from ellipse fitting and a projection histogram along the axes of the ellipse are used as the features for distinguishing different postures of the human. These features are then fed into a directed acyclic graph support vector machine (DAGSVM) for posture classification, the result of which is then combined with derived floor information to detect a fall. From a dataset of 15 people, we show that our fall detection system can achieve a high fall detection rate (97.08%) and a very low false detection rate (0.8%) in a simulated home environment.

Outdoor color rating of sweet cherries using computer vision

In this paper, we report the results of an exploration study of the feasibility of using computer vision to conduct accurate color rating of sweet cherry in outdoor orchard environments. Pre-harvest color rating of cherry is important to growers in determining the optimal harvest time. Currently, the in-field rating relies heavily on the manual comparison between the color of cherry fruits and standard color charts, which is both labor intensive and subjective. It is not uncommon to have one or two grades of deviation. A computer vision-based color rating system was developed in an attempt to provide an automatic and objective way to achieve more consistent and accurate color ratings. This system successfully used a camera flash to reduce the effects of two major obstacles in outdoor color rating: (1) inconsistent ambient light; and (2) glaring reflections on cherry skin. To mimic the manual color rating practice that is widely accepted by sweet cherry growers today, a task-oriented image processing algorithm was developed to remove the glaring reflections and to classify the color of cherries into seven levels. Field tests showed that the overall accuracy of the rating exceeded 85% based on 660 samples from three field tests under natural, outdoor lighting conditions. The tests validated the feasibility of using a computer vision system to achieve accurate and objective color ratings of sweet cherry under outdoor natural light conditions for actual in-orchard use.

Virtual Volumetric Graphics on Commodity Displays Using 3D Viewer Tracking

Three dimensional (3D) displays typically rely on stereo disparity, requiring specialized hardware to be worn or embedded in the display. We present a novel 3D graphics display system for volumetric scene visualization using only standard 2D display hardware and a pair of calibrated web cameras. Our computer vision-based system requires no worn or other special hardware. Rather than producing the depth illusion through disparity, we deliver a full volumetric 3D visualization--enabling users to interactively explore 3D scenes by varying their viewing position and angle according to the tracked 3D position of their face and eyes. We incorporate a novel wand-based calibration that allows the cameras to be placed at arbitrary positions and orientations relative to the display. The resulting system operates at real-time speeds (~25 fps) with low latency (120---225 ms) delivering a compelling natural user interface and immersive experience for 3D viewing. In addition to objective evaluation of display stability and responsiveness, we report on user trials comparing users' timings on a spatial orientation task.

Efficient focus assessment for a computer vision-based Vickers hardness measurement system

A large variety of computationally lightweight functions used for assessing image sharpness in the spatial domain is evaluated for application in a passive autofocus system in the context of microindentation-based Vickers hardness testing. Alternatively, the file size of compressed JPEG images is proposed to determine image sharpness. The functions are evaluated on a significant dataset of microindentation images with respect to their properties required for focus search and sharpness assessment, their robustness to downsampling the image data, and their computational demand. Experiments suggest that of all spatial domain techniques considered, the simple Brenner autofocus function is the best compromise between accuracy and computational effort, while JPEG file size is a versatile solution when the application context allows.

An engineered approach to stem cell culture: automating the decision process for real-time adaptive subculture of stem cells.

Current cell culture practices are dependent upon human operators and remain laborious and highly subjective, resulting in large variations and inconsistent outcomes, especially when using visual assessments of cell confluency to determine the appropriate time to subculture cells. Although efforts to automate cell culture with robotic systems are underway, the majority of such systems still require human intervention to determine when to subculture. Thus, it is necessary to accurately and objectively determine the appropriate time for cell passaging. Optimal stem cell culturing that maintains cell pluripotency while maximizing cell yields will be especially important for efficient, cost-effective stem cell-based therapies. Toward this goal we developed a real-time computer vision-based system that monitors the degree of cell confluency with a precision of 0.791±0.031 and recall of 0.559±0.043. The system consists of an automated phase-contrast time-lapse microscope and a server. Multiple dishes are sequentially imaged and the data is uploaded to the server that performs computer vision processing, predicts when cells will exceed a pre-defined threshold for optimal cell confluency, and provides a Web-based interface for remote cell culture monitoring. Human operators are also notified via text messaging and e-mail 4 hours prior to reaching this threshold and immediately upon reaching this threshold. This system was successfully used to direct the expansion of a paradigm stem cell population, C2C12 cells. Computer-directed and human-directed control subcultures required 3 serial cultures to achieve the theoretical target cell yield of 50 million C2C12 cells and showed no difference for myogenic and osteogenic differentiation. This automated vision-based system has potential as a tool toward adaptive real-time control of subculturing, cell culture optimization and quality assurance/quality control, and it could be integrated with current and developing robotic cell cultures systems to achieve complete automation.

Computer Vision-Based Augmented Reality System for Assembly Interaction

A research effort aimed at creating a computer vision-based augmented reality assembly system is presented. The architecture of this system for assembly interaction is described. To realize intuitive human-computer interaction, a computer vision-based hand tracking and gesture recognition approach is applied. This approach uses the color-based method to segment the hand from background, and locate the hand position by the marker attached on hand, and then recognizes the gesture according to the geometry constraint of human hand. In comparison with the traditional approaches, this new method does not require a stationary camera, and is not sensitive with intensity difference. So, it provides a real time performance and is easy to realize. Moreover, occlusion identification is studied in this paper to raise the real and virtual objects combination effect. Finally, a prototype system is provided to demonstrate the effectiveness and robustness of the presented approaches.

An adaptive, real-time, traffic monitoring system

In this paper we describe a computer vision-based traffic monitoring system able to detect individual vehicles in real-time. Our fully integrated system first obtains the main traffic variables: counting, speed and category; and then computes a complete set of statistical variables. The objective is to investigate some of the difficulties impeding existing traffic systems to achieve balanced accuracy in every condition; i.e. day and night transitions, shadows, heavy vehicles, occlusions, slow traffic and congestions. The system we present is autonomous, works for long periods of time without human intervention and adapts automatically to the changing environmental conditions. Several innovations, designed to deal with the above circumstances, are proposed in the paper: an integrated calibration and image rectification step, differentiated methods for day and night, an adaptive segmentation algorithm, a multistage shadow detection method and special considerations for heavy vehicle identification and treatment of slow traffic. A specific methodology has been developed to benchmark the accuracy of the different methods proposed.

Virtual Slide Guitar

This article describes a system for virtual slide guitar playing. From the control point of view, it can be seen as a successor of the virtual air guitar (VAG) developed at Helsinki University of Technology (TKK) a few years ago (Karjalainen et al. 2006). The original VAG allows the user to control an electric guitar synthesizer by mimicking guitar-playing gestures. In the current work, the same gesture-control approach that was found successful in the VAG is used: a computer-visionbased system, in which a camera detects the player's hands and a computer tracks the hand movements and converts them into control data, such as pluck events and string length. Sound synthesis in the virtual slide guitar application is based on an energy-compensated time-varying digital waveguide model of a guitar with new extensions to generate contact sounds caused by the slide tube touching the strings. Video files showing the virtual slide guitar in action can be found on the accompanying Web page (www.acoustics.hut.fi/publications/papers/vsg/) and the forthcoming 2008 Computer Music Journal Sound and Video Anthology DVD. Although the current implementation uses a camera-based user interface, it can also be controlled by other humanmachine interfaces or computer programs. Excellent reviews on gestural control of music synthesis have been written by Paradiso (1997) and by Wanderley and Depalle (2004). Camera-based gesture analysis has become very sophisticated owing to increased computing power and new methodologies provided by research, as exemplified by the EyesWeb platform (Camurri et al. 2000, 2005; Gorman et al. 2007). Digital waveguide modeling is mature technology, which can be applied to high-quality synthesis of various musical instruments (Smith 1992; Valimaki et al. 2006). For more information on waveguide synthesis of string instruments, see Valimaki et al. (1996), Karjalainen, Valimaki, and Tolonen (1998), and Karjalainen et al. (2006). The roots of this study are in an early prototype of the TKK virtual air guitar, which here is called the rubber-band virtual air guitar. It was a simplified gesture-control system, in which the distance between the hands was detected and di-

Rationale, Design and Implementation of a Computer Vision-Based Interactive E-Learning System

This article presents a schematic application of computer vision technologies to e-learning that is synchronous, peer-to-peer-based, and supports an instructor’s interaction with non-computer teaching equipments. The article first discusses the importance of these focused e-learning areas, where the properties include accurate bidirectional interaction and low cost hardware; system portability and versatile vision technology are emphasized. In the subsequent sections, we present some results aiming to achieve these goals. In particular, we highlight the most recent advancements in the interactive PTZ camera control from both the instructor and remote student. We also illustrated how these results have successfully addressed the challenges.

Performance characterization of the dynamic programming obstacle detection algorithm

A computer vision-based system using images from an airborne aircraft can increase flight safety by aiding the pilot to detect obstacles in the flight path so as to avoid mid-air collisions. Such a system fits naturally with the development of an external vision system proposed by NASA for use in high-speed civil transport aircraft with limited cockpit visibility. The detection techniques should provide high detection probability for obstacles that can vary from subpixels to a few pixels in size, while maintaining a low false alarm probability in the presence of noise and severe background clutter. Furthermore, the detection algorithms must be able to report such obstacles in a timely fashion, imposing severe constraints on their execution time. For this purpose, we have implemented a number of algorithms to detect airborne obstacles using image sequences obtained from a camera mounted on an aircraft. This paper describes the methodology used for characterizing the performance of the dynamic programming obstacle detection algorithm and its special cases. The experimental results were obtained using several types of image sequences, with simulated and real backgrounds. The approximate performance of the algorithm is also theoretically derived using principles of statistical analysis in terms of the signal-to-noise ration (SNR) required for the probabilities of false alarms and misdetections to be lower than prespecified values. The theoretical and experimental performance are compared in terms of the required SNR.

Computer Vision-Aided Fabric Inspection System for On-Circular Knitting Machine

This paper describes a computer vision-based fabric inspection system implemented on a circular knitting machine to inspect the fabric under construction. The study consisted of two parts. In the first part, detection of defects in knitted fabric was performed and the performance of three different spectral methods, namely the discrete Fourier transform, the wavelet and the Gabor transforms were evaluated off-line. In the second part, knitted fabric defect-detection and classification was implemented on-line. The captured images were subjected to a defect-detection algorithm, which was based on the concepts of the Gabor wavelet transform, and a neural network (as a classifier). An operator encountering defects also evaluated the performance of the system. The fabric images were broadly classified into seven main categories as well as seven combined defects. The results of the designed system were compared with those of human vision.

Design and prototype development of a computer vision-based lumber production planning system

Abstract A computer vision-based lumber production planning system (CVLPPS) is described. Computer axial tomography (CT or CAT) images of hardwood logs are analyzed for identification and classification of internal log defects. Individual CT image slices are analyzed for detection of 2D defects which are correlated across CT image slices in order to establish 3D support and identify true 3D defects. CVLPPS is capable of 3D reconstruction and rendering of the log and its internal defects from the individual CT image slices. It is also capable of simulation and rendering of key machining operations such as sawing and veneering on the 3D reconstructions of the logs. From the 3D reconstruction of the log and knowledge of its internal defects, CVLPPS can formulate sawing strategies to optimize the yield and grade of the resulting lumber. A prototype CVLPPS was developed and tested on CT images of hardwood logs from White Ash, Hard Maple, Red Oak and Black Walnut. Experimental results showed that CVLPPS could identify and localize a large majority of the internal log defects and thereby result in a 23–63% gain in value yield recovery when compared to a lumber processing strategy that did not use internal log structure information. Issues pertaining to the deployment of CVLPPS in a real-time lumber production environment are discussed. It is shown that the various components of CVLPPS are amenable to parallel and distributed computing, thus making a real-time implementation of CVLPPS practically feasible. CVLPPS could also be used as a decision aid for lumber production planning and an interactive training tool for novice sawyers and machinists.

Computer Vision Detection System for Network Model Validation

A computer vision-based traffic detection system has been developed and tested that is capable of uniquely identifying vehicles in a traffic network from video images and later reidentifying them at other detection sites in the network. This capability may serve as the basis for deterministic validation of microscopic network flow models, origin-destination tables, and travel-time tables. The system is referred to as the video-based vehicle signature analysis and tracking system. With video cameras used as primary sensors, detection modules generate a numeric video signature vector (VSV) for each vehicle and transmit them to a central Internet-connected correlation computer via a public low-power wireless network. The correlation computer attempts to match vehicles, as represented by their VSVs, generated at successive detection sites to enable a sampled real-time microscopic flow representation of a freeway network. Test results indicate 93.6 percent accuracy in correctly reidentifying vehicles at successive sites. The tendency of the system to incorrectly match different vehicles at successive sites was measured at 0.0116 percent. Potential advantages are low cost in widespread deployment, simplicity and reliability of detection, minimal bandwidth and storage requirements for transmission of the VSV, and reasonable identification ability without violation of privacy rights. The primary disadvantages are the requirement to place video cameras above each traffic lane and video imaging limitations related to reliance on ambient illumination.

A SHAPE CLASSIFIER BY USING IMAGE PROJECTION AND A NEURAL NETWORK

In this research a computer vision-based shape recognition system which combines an image projection and the back propagation neural network is proposed. The system uses the image projection and features of the projection histogram to represent the shape, and uses the back propagation neural network to classify the shape. In this research, a fast normalization scheme is developed which is especially designed for image projection and can also find the same projection axis no matter what the object's orientation. Therefore, the proposed system can be invariant to scale, translation, and orientation variations. This research finds that the direct representation scheme of the back propagation neural net has better accuracy than the binary coding representation scheme. This research also finds that the proposed system's classification accuracy is greater than 98.8% and its processing speed is also very fast. Therefore, the proposed system is feasible for use in shape classification, in practice.

A Computer Vision-Based Shape-Classification System UsingImage Projection and a Neural Network

In this research, a computer vision-based shape-recognition system which combines an image projection and the back propagation neural network is proposed. The system uses the image projection and features of the projection histogram to represent the shape, and uses the back propagation neural network to classify the shape. In this research, a fast normalisation scheme is developed which is especially designed for the image projection. It can find the same projection axis no matter how the object is oriented. Therefore, the proposed system can be invariant to scale, translation, and orientation variations. This research finds that the direct representation scheme of the back propagation neural net has better accuracy than the binary coding representation scheme. This research also finds that classification accuracy of the proposed system is greater than 98.8% and its processing speed is also very fast. Therefore, the proposed system is feasible for use in shape classification.

Live cell image segmentation.

A major requirement of an automated, real-time, computer vision-based cell tracking system is an efficient method for segmenting cell images. The usual segmentation algorithms proposed in the literature exhibit weak performance on live unstained cell images, which can be characterized as being of low contrast, intensity-variant, and unevenly illuminated. We propose a two-stage segmentation strategy which involves: 1) extracting an approximate region containing the cell and part of the background near the cell, and 2) segmenting the cell from the background within this region. The approach effectively reduces the influence of peripheral background intensities and texture on the extraction of a cell region. The experimental results show that this approach for segmenting cell images is both fast and robust.

Automatic spacecraft docking using computer vision-based guidance and control techniques

An innovative approach to automatic spacecraft docking using a computer vision-based control system is introduced. Precision control of the relative spacecraft velocity is required to achieve docking with a docking platform on a space station or on another spacecraft. We propose use of a computer vision system as a position and orientation sensor for obtaining feedback information used by guidance and control loops. A camera, fixed to the spacecraft, tracks a standard rhombus mark fixed on the docking platform. Discrete-time position and orientation estimates of the spacecraft, relative to a coordinate frame fixed to the docking platform, are obtained by solving a constrained nonlinear least-squares problem and are used by the spacecraft feedback control loops. The accuracy of the computer vision estimates improves as the relative range decreases, thereby providing improved feedback information when it is most critical. Feedback control loops for the spacecraft, using three pairs of gas jet thrusters, are suggested that keep the camera always pointed at the rhombus mark and that perform precise control of the spacecraft range to achieve soft docking. The interactions between the vision system and the control systems are emphasized. Computer simulations of an integrated docking system verify the practical feasibility of this proposed automatic docking approach.