Kanade-Lucas-Tomasi Research Articles

MRI Images under the Optimized Registration Algorithm for Primary Open Angle Glaucoma Visual Path Damage

To explore the impact of different image registration algorithms on the diagnosis of visual path damage in patients with primary open angle glaucoma (POAG), 60 cases of suspected POAG patients were selected as the research objects. Shape-preserving scale invariant feature transform (SP-SIFT) algorithm, scale invariant feature transform (SIFT) algorithm, and Kanade-Lucas-Tomasi (KLT) algorithm were compared and applied to MRI images of 60 POAG patients. It was found that the SP-SIFT algorithm converged after 33 iterations, which had a higher registration speed than the SIFT algorithm and the KLT algorithm. The mean errors of the SP-SIFT algorithm in the rotation angle, X-direction translation, and Y-direction translation were 2.11%, 4.56%, and 4.31%, respectively. Those of the SIFT algorithm were 5.55%, 9.98%, and 7.01%, respectively. Those of the KLT algorithm were 7.45%, 11.31%, and 8.56%, respectively, and the difference among algorithms was significant ( P < 0.05 ). The diagnostic sensitivity and accuracy of the SP-SIFT algorithm for POAG were 96.15% and 94.34%, respectively. Those of the SIFT algorithm were 94.68% and 90.74%, respectively. Those of the KLT algorithm were 94.21% and 90.57%, respectively, and the three algorithms had significant differences ( P < 0.05 ). The results of MRI images based on the SP-SIFT algorithm showed that the average thickness of the cortex of the patient’s left talar sulcus, right talar sulcus, left middle temporal gyrus, and left fusiform gyrus were 2.49 ± 0.15 mm, 2.62 ± 0.13 mm, 3.00 ± 0.10 mm, and 2.99 ± 0.17 mm, respectively. Those of the SIFT algorithm were 2.51 ± 0.17 mm, 2.69 ± 0.12 mm, 3.11 ± 0.13 mm, and 3.09 ± 0.14 mm, respectively. Those of the KLT algorithm were 2.35 ± 0.12 mm, 2.52 ± 0.16 mm, 2.77 ± 0.11 mm, and 2.87 ± 0.17 mm, respectively, and the three algorithms had significant differences ( P < 0.05 ). In summary, the SP-SIFT algorithm was ideal for POAG visual pathway diagnosis and was of great adoption potential in clinical diagnosis.

Validation test on 3d heart phantom for mitral valve leaflet tracking

<p><span style="font-family: 'Times New Roman',serif; font-size: 9pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: EN-US; mso-bidi-language: AR-SA;" lang="EN-US"><span style="font-family: 'Times New Roman',serif; font-size: 9pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: EN-US; mso-bidi-language: AR-SA;" lang="EN-US">Mitral valve movement is essential to be identified in order to monitor the abnormality of blood flow in right side of heart. The estimation and tracking of mitral valve has seldom been investigated since it required high temporal rate to scan the echocardiography images and it depends on the operator to capture the low-speckle and-noise images. This study presents the validation experiment performed on heart phantom made of t</span><span style="font-family: 'Times New Roman',serif; font-size: 9pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: EN-US; mso-bidi-language: AR-SA;" lang="EN-US">hermoplastic polyurethane (TPU) filament which the objective is to validate the previous </span><span style="font-family: 'Times New Roman',serif; font-size: 9pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: EN-US; mso-bidi-language: AR-SA;" lang="EN-US">features tracking technique implemented in mitral valve locating in video frames using Kanade-Lucas-Tomasi (KLT) algorithm. The outcome was able to automatically detect the edge of mitral valve and thus in future, it manages to predict the flowing of blood pattern. An in-vitro experiment was conducted which involved a valve phantom scanning in water tank that connected to water pump. It was found in this study that the technique capable to detect and visualize the mitral valve up to 59 frames in 2.36 secondsby tracking the features of minimum eigenvalue within the selected region. It was also produced a good agreement of valve distance between the true value and the measured one, which achieved the minimum of 88% similarity. This yielded the validation of the proposed technique to track and visualize the mitral valves. </span></span></p>

Computer vision‐based real‐time cable tension estimation in Dubrovnik cable‐stayed bridge using moving handheld video camera

Cables are essential components of the cable-stayed bridges as they serve as the main load-bearing component. Hence, continuous monitoring of such cables becomes necessary as they are vulnerable to the fatigue damage induced by dynamic loads. Sensors are attached to the cables to examine the health of the cables; however, these contact-based sensors can malfunction in harsh weather condition, which makes impossible to estimate the cable health in such unfavorable condition. Therefore, in this paper, we propose a completely noncontact video-based stay-cable tension measurement technique where the video is recorded using a moving handheld camera at a significant distance from the structure itself. Here, the cable tension is determined from vibration-based measurement, but the vibration of the cable recorded in the video includes the true vibration of the cable along with the camera motion. Hence, we amalgamated a series of image processing techniques to nullify the camera movement. First, we detect the camera movement based on the movement of the bridge deck and pylon, which are fixed objects, using Kanade–Lucas–Tomasi (KLT) feature tracking algorithm. Then we nullify the camera movement by using the affine transformation matrix obtained by random sample consensus (RANSAC) algorithm. Subsequently from the steady video, the cable motions are estimated using the phase-based motion estimation technique. From the time history of the cable vibration, real-time frequency variations are estimated using Short-Time Fourier Transform (STFT). Finally, the real-time tension is determined from this dominant frequency variation history using the taut-string theory. This paper shows the significant potential of camera-based sensing techniques in structural health monitoring as the mean estimated tension and the design cable tension are found to be comparable.

Micro-expression recognition based on motion detection method

Micro-expressions are emotional representations that occur spontaneously and cannot be controlled by humans. The micro-expression movements are temporary with fast duration and have subtle movements with little intensity. This is difficult to detect with the human eye. Previous studies have shown that micro-expression movements occur in several areas of the face. This study aims to determine the subtle movements in several areas of the face using the motion detection method. We compared the performance of two motion detection methods: the optical flow method and the Block Matching Algorithm (BMA) method. The optical flow method uses the Kanade-Lucas Tomasi (KLT) method and the BMA method uses the Phase Only Correlation (POC) algorithm. Observations were carried out based on region, where the face area was divided into several observation areas: eyebrows, eyes and mouth. Both methods perform motion detection between frames. The KLT method tracks the movement of the observation points on the frame movement. Meanwhile, the POC method matches the blocks between frames. If the two blocks are the same, no motion vector is generated. However, if the two blocks are different, it is assumed that there is a translational motion and a motion vector is generated. Experiments were conducted using a dataset from CASME II with emotional classes of disgust, happiness, surprise, and sadness. The classification accuracy of the POC method is 94% higher than the KLT method of 84.8% which uses the SVM classification.

Optical Flow Monocular Visual-Inertial Odometry with Online Photometric Calibration

Kanade-Lucas-Tomasi (KLT) optical flow algorithm based on the brightness constancy assumption is widely used in visual simultaneous localization and mapping (SLAM) and visual odometry (VO). However, the automatic adjustment of camera exposure time, the attenuation factor of sensor irradiance caused by vignetting, and the nonlinear camera response function will cause the same feature point to have different brightness values on different image frames, thus breaking this assumption. Hence, we propose a gain-adaptive KLT optical flow algorithm with online photometric calibration, and on this basis, design a monocular visual-inertial odometry which is insensitive to brightness changes. This method can calibrate the photometric parameters online in real time, meet the assumption of constant brightness in practical applications, and make the algorithm more robust and accurate in the case of dynamic changes in brightness. Experimental results on the TUM Mono and EuRoC datasets show that the proposed algorithm can reliably calibrate the photometric parameters of any video sequence and perform well in the environment with varying brightness.

Robust Real-Time Traffic Surveillance with Deep Learning.

Real-time vehicle monitoring in highways, roads, and streets may provide useful data both for infrastructure planning and for traffic management in general. Even though it is a classic research area in computer vision, advances in neural networks for object detection and classification, especially in the last years, made this area even more appealing due to the effectiveness of these methods. This study presents TrafficSensor, a system that employs deep learning techniques for automatic vehicle tracking and classification on highways using a calibrated and fixed camera. A new traffic image dataset was created to train the models, which includes real traffic images in poor lightning or weather conditions and low-resolution images. The proposed system consists mainly of two modules, first one responsible of vehicle detection and classification and a second one for vehicle tracking. For the first module, several neural models were tested and objectively compared, and finally, the YOLOv3 and YOLOv4-based network trained on the new traffic dataset were selected. The second module combines a simple spatial association algorithm with a more sophisticated KLT (Kanade–Lucas–Tomasi) tracker to follow the vehicles on the road. Several experiments have been conducted on challenging traffic videos in order to validate the system with real data. Experimental results show that the proposed system is able to successfully detect, track, and classify vehicles traveling on a highway on real time.

Detection and Tracking of Face Location in the Pre-processing Stage of Recognition of Micro Expressions Using the Kanade-Lucas-Tomasi (KLT) Method

Research and observation of micro-expression movements requires a careful pre-processing stage, because it is associated with observing subtle movements and very fast durations. At this stage, the detection and tracking of the face area must always be precise so that the observation of movement in the face area can be accurate. Several video samples of the micro expression dataset showed facial movements followed by small movements of the head. The movements outside the movement in this facial area will affect the performance of the micro expression recognition system. This study detects and tracks the location of faces during micro-expression movements. The Viola-Jones method is used for face detection and the Kanade Lucas Tomasi (KLT) method for tracking feature points. After the face is detected correctly, a point tracking is performed at the tip of the nose. This tracking of the movements at the nose tip is followed by the formation of the facial area. So that the face area will always be in the right position even if there is movement in the head. The test and testing data used in this study were the CASME II dataset. The test from this research shows that the detection and tracking of facial areas can be done precisely and accurately on all test data. It is hoped that with the formation of facial areas that are always precisely tracked, the next process of recognition of micro expressions can run well.

Face Detection By some Methods based on MATLAB

Over the past few years, technology has begun to discover things rapidly growing, as this technology has come to dominate the vision of computers, which have taken an important role in monitoring cars and their safety.
 In this work, the face detection and tracking process was highlighted,[6], and two algorithms were created to clarify the role of the MATLAB program in creating a simple system for facial tracking and detection where the first example shows the application of Algorithm1 using the CAMShift method where the face was tracked where the results were weak due to sufficient contrast between Face and background area, the algorithm1 was created to implement the method, [2].
 Example2 is using the method Kanade Lucas Tomasi (KLT), which is a simple system where it is able to detect the face and track it where the system is able to detect the face when the head moves and tilts, with Estim Geometric Transform (EGT) function and Duration Geometric Transform (DGT), The algorithm 2 was created to implement the method.
 The examples in this work show the implementation and efficiency of the mentioned algorithms

A marker-free method for structural dynamic displacement measurement based on optical flow

Information on dynamic displacement is an effective indicator for structures condition evaluation and provides a quantised insight into the structural analysis. This paper proposes a low-cost system based on computer vision using a smartphone to measure dynamic displacement, and then identify the dynamic properties of structures. Conventional sensors like linear variable differential transformers (LVDT), GPS, and accelerometer in monitoring systems have limitations of high price, inaccessibility, and accuracy. However, some new technologies eliminate these disadvantages. For example, a smartphone with a high-resolution camera becomes more affordable, which is regarded as appropriate equipment for structural health monitoring (SHM). Based on the optical flow, this method allows users to track points with a specific interval in the chosen region and reduces the displacement drift induced by the Kanade–Lucas–Tomasi (KLT) method. With the method applied, the region of interest is relocated according to pixel motion, and feature points are reselected. The accuracy of the system is verified on a laboratory suspension bridge model, and the results of modal frequency are confirmed with an accelerometer and FEM simulations.

Human motion tracking and positioning for augmented reality

AR (Augmented reality) is a research hotspot in the current computer application field. AR technology enhances people’s understanding and experience of the real environment by adding virtual objects to real scenes to integrate virtual objects with the real environment. Aiming at the weak processing power of intelligent terminals and the characteristics of limited hardware resources, this paper proposes a more effective human motion feature extraction and descriptor algorithm. The feature point detection and positioning method suitable for intelligent terminals is proposed in a targeted manner, which solves the problem of mismatching of similar structures. In addition, this paper proposes an AR-oriented recursive tracking algorithm for human motion. The positional relationship of the current frame is calculated from the position of the previous frame. A combination of ORB (Oriented fast and Rotated Brief) feature descriptors and KLT (Kanade-Lucas-Tomasi) algorithm is adopted. The ORB feature descriptor matched by the first frame image and the reference image is tracked by the KLT tracking algorithm, and the feature descriptor of the previous frame is tracked in the current frame, thereby eliminating the phenomenon of virtual object jitter. The experimental results show that the recursive tracking scheme has better performance in time and precision than the detection tracking scheme.

A novel framework for UAV returning based on FPGA

To date, most unmanned aerial vehicle (UAV) returning technology has relied on the global positioning system (GPS). The risk is that the UAV may be spoofed by fake GPS signals, which could cause it to deviate from its expected flight route. Therefore, a returning framework without GPS is particularly essential. To address this issue, this paper proposes a new UAV returning framework based on improved Kanade–Lucas–Tomasi (KLT) feature tracker. This framework addresses the issues of high computational complexity of KLT by using a field-programmable gate array and designs the hardware acceleration architecture by integrating several optimization methods. Moreover, it adopts hardware/software co-design technology to improve parallelism and resource utilization. With these optimizations, the framework can be deployed on most development boards with flexible hardware resources. Finally, the effectiveness of the improved algorithm are demonstrated using zedboard development board, and the results show that the processing speed can achieve 60 frames per second (fps).

Optical flow refinement using iterative propagation under colour, proximity and flow reliability constraints

This study proposes a strategy to refine optical flow based on the estimated reliability maps. These maps are firstly estimated a posteriori after the motion estimation by the well-known Kanade–Lucas–Tomasi (KLT). With two new defined criteria based, respectively, on the optical flow local variance and the temporal evolution of the KLT residuals, a global refinement of the motion map is then carried out through two stages under the control of the reliability measures and the colour local homogeneousness. According to the experiments performed on the Middlebury dataset, the authors' reliability measures prove to be a good indicator for the quality of the estimation. Indeed, the correction process increases the global reliability measures and reduces the global errors in a significant way. The experiments show that the quality is higher than classical estimation methods and ranked at 88/168 on Middlebury website

A deformation detection method for aircraft skin on uniform pressure by using speckle image correlation technology

The fatigue cracks of aircraft skin which are caused by the pressurization for a civilian airplane during the flight process will bring potential serious safety risks. Therefore, it is necessary to study the deformation mechanism for the aircraft skin on uniform pressure. Traditional fatigue testing machines can only be used for simple tension and compression tests which cannot be applied directly. This paper proposes a deformation detection method by using speckle image correlation technology. Firstly, a fatigue testing machine simulating uniform pressure has been designed. Secondly, a speckle image correlation method by using improved KLT (Kanade-Lucas-Tomasi) algorithm is presented to obtain the coordinates of the coated speckle spots. Finally, static and dynamic experiments were implemented to verify the accuracy and effectiveness of the proposed method. In conclusion, this method can be applied in airworthiness certification to improve the testing efficiency.

Classification of Facial Nerve Paralysis Based on Regional Evaluation

This paper present an approach of regional evaluation using Kanade - Lucas – Tomasi (KLT) method and extract feature points to classify the normal and patients subjects and also to determine the severity level of paralysis for each region of face. There are Individual Score Table, Total Score Table and Paralysis Score Table were presented. A 100% of accuracy has been obtained in identifying the paralysis with k =3 using k-NN classifier for the Individual Score.

An integrative approach for path planning and tracking of shape aware mobile robot in structured environment using vision sensor

A shape-aware path planning algorithm is necessary for real-time execution of a task by a mobile robot whereas path planning algorithms available in literature consider mobile robot as a point object. Current work proposes a shape-aware A* path planning approach with a heuristic function to accommodate the shape of mobile robot. In this paper, the detection, tracking and control of the robot has been carried out for the mobile robot while performing a task in a structured environment. For the implementation and validation, an overhead camera is used to capture the images of obstacles in the task space and published in ROS platform. The captured images are also processed using OpenCV software for detection and tracking using Kanade-Lucas-Tomasi (KLT) and Kalman filter algorithms for different test scenarios. The proposed approach accurately detects and tracks the shape aware mobile robot with percentage error ranging from 6%-10% in different cases.

A fast hybrid computer vision technique for real-time embedded bus passenger flow calculation through camera

Bus passenger flow calculation system is a critical part of the smart public transportation framework. Bus passenger flow information can help to make data statistics report of the passenger at a bus station which can be used by public transport operator to evaluate the quality of the transportation. Statistics report of crowded passengers in the bus station help managers to understand the bus transit operations, can provide the database for the intelligent transportation scheduling, help to provide more and better services for passengers, overall data statistics of passengers has important practical significance to improve public transport environment. This paper presents a passenger counting algorithm based on hybrid machine learning approach. In the first step, an advanced method is used to extract the Histogram of oriented gradients (HOG) feature of passenger’s heads. Classification of head features is done by using support vector machine (SVM) as a classifier for the liner model. Heads are detected successfully after performing all steps. In next step Kanade-Lucas-Tomasi (KLT) is used to reality head tracking, the multiple target tracking is achieved and the head motion trajectory of passenger target is captured stably. At last, the trajectory is analyzed and the automatic counting of bus passenger flow is realized. In the last step, the proposed algorithm is move to embedded system for practical implementation. In this paper, the algorithm intends to use ADSP-BF609 embedded platform for transplantation. The experimental results demonstrate that the statistical accuracy of the proposed algorithm is enhanced successfully; especially during the daytime with the good illustration, the effective counting of the passenger flow is achieved and the inward and outward passenger counting can be realized. In this paper three feature extraction models are used namely local binary patterns, histograms of oriented gradients and binarized statistical image in order to get accurate features. Furthermore, three common classification techniques including naive bayes classifier, boosted tress and support vector machines are used for fine classification of extracted vectors obtained from different features extractors model. 94.50% accuracy is achieved when support vector machine (SVM) classifies the features extracted using Histogram of oriented gradients (HOG). SVM surpasses the accuracy obtained by Boosted tree namely 81.30% using Histogram of oriented gradients (HOG) features.

Human Behavior Understanding in Big Multimedia Data Using CNN based Facial Expression Recognition

Human behavior analysis from big multimedia data has become a trending research area with applications to various domains such as surveillance, medical, sports, and entertainment. Facial expression analysis is one of the most prominent clues to determine the behavior of an individual, however, it is very challenging due to variations in face poses, illuminations, and different facial tones. In this paper, we analyze human behavior using facial expressions by considering some famous TV-series videos. Firstly, we detect faces using Viola-jones algorithm followed by tracking through Kanade-Lucas-Tomasi (KLT) algorithm. Secondly, we use histogram of oriented gradients (HOG) features with support vector machine (SVM) classifier for facial recognition. Next, we recognize facial expressions using the proposed light-weight convolutional neural network (CNN). We utilize data augmentation techniques to overcome the issue of appearance of faces from different views and lightening conditions in video data. Finally, we predict human behaviors using an occurrence matrix acquired from facial recognition and expressions. The subjective and objective experimental evaluations prove better performance for both facial expression recognition and human behavior understanding.

Integrated features and GMM Based Hand Detector Applied to Character Recognition System under Practical Conditions

Detection of bare-hand under non-ideal conditions is a challenging task. Most of the existing hand detection systems are developed under limited environmental constraints. In this study, a robust two-level bare-hand detector is integrated with a 58 keyboard characters recognition model. At first, the Gaussian mixture model (GMM) based foreground detector is used to segment the region of interest (ROI), which is further classified using Color-texture and texture based models to detect the actual fist. The detected hand is tracked using modified Kanade–Lucas–Tomasi (KLT) tracker to generate the required trajectory points of the character. The feature space for character recognition consists of existing features and three new features, namely, Local Geometrical Area Ratio (LGAR), Area of two halves (ATH), Curve-Area feature (CAF) that are extracted from the trajectory points. Feature space is optimized using statistical analysis algorithms. Multi-factor analysis of individual character subsets such as alphabets, numbers, ASCII characters, etc., are carried out using multiple conventional classifiers along with Support vector machine (SVM), extreme learning machine (ELM), artificial neural network (ANN), and proposed Neuro-fuzzy classifiers. The proposed GMM based motion detection method achieves an accuracy of 100% during the segmentation of ROI, followed by an increase of 46.77% in the accuracy of two-level hand detection under non-ideal conditions. Maximum accuracy of 58 character system using proposed features and ANN classifier is observed to be 92.56%.

Real-Time Downward View Generation of a Vehicle Using Around View Monitor System

In this paper, a method to generate the downward image of current vehicle location using a commercial around view monitor (AVM) is proposed. The proposed system consists of three stages, namely feature tracking, obstacle filtering, and downward view generation. In the feature tracking stage, the Shi–Tomasi corner detection is used and feature tracking is performed with a Kanade–Lucas–Tomasi (KLT) tracker to determine the transformation between AVM images. In the obstacle filtering stage, features on obstacles are filtered by using the difference in a tracking distance between features detected on the ground and features detected on the obstacle. This is performed by using a histogram based on the tracking distance of features. Finally, in the downward view generation stage, transformation between the current AVM image and the previous AVM image is obtained by using the tracked feature pair refined via the aforementioned steps. Specifically, the random sample consensus (RANSAC) method is used to obtain transformations from which the influence of outliers is removed. The downward image of the current vehicle is generated by the obtained transformation and is synthesized to the existing AVM image. The results indicate that the proposed system synthesizes the existing AVM image and the generated downward image of a vehicle in a seamless way by determining the exact pair of matching points between current and the previous AVM images. We believe the proposed system can be utilized in efficient wireless charging system and safe driving system.

Vision-Based Measurement of Heart Rate from Ballistocardiographic Head Movements Using Unsupervised Clustering.

Heart rate has been measured comfortably using a camera without the skin-contact by the development of vision-based measurement. Despite the potential of the vision-based measurement, it has still presented limited ability due to the noise of illumination variance and motion artifacts. Remote ballistocardiography (BCG) was used to estimate heart rate from the ballistocardiographic head movements generated by the flow of blood through the carotid arteries. It was robust to illumination variance but still limited in the motion artifacts such as facial expressions and voluntary head motions. Recent studies on remote BCG focus on the improvement of signal extraction by minimizing the motion artifacts. They simply estimated the heart rate from the cardiac signal using peak detection and fast fourier transform (FFT). However, the heart rate estimation based on peak detection and FFT depend on the robust signal estimation. Thus, if the cardiac signal is contaminated with some noise, the heart rate cannot be estimated accurately. This study aimed to develop a novel method to improve heart rate estimation from ballistocardiographic head movements using the unsupervised clustering. First, the ballistocardiographic head movements were measured from facial video by detecting facial points using the good-feature-to-track (GFTT) algorithm and by tracking using the Kanade–Lucas–Tomasi (KLT) tracker. Second, the cardiac signal was extracted from the ballistocardiographic head movements by bandpass filter and principal component analysis (PCA). The relative power density (RPD) was extracted from its power spectrum between 0.75 Hz and 2.5 Hz. Third, the unsupervised clustering was performed to construct a model to estimate the heart rate from the RPD using the dataset consisting of the RPD and the heart rate measured from electrocardiogram (ECG). Finally, the heart rate was estimated from the RPD using the model. The proposed method was verified by comparing it with previous methods using the peak detection and the FFT. As a result, the proposed method estimated a more accurate heart rate than previous methods in three experiments by levels of the motion artifacts consisting of facial expressions and voluntary head motions. The four main contributions are as follows: (1) the unsupervised clustering improved the heart rate estimation by overcoming the motion artifacts (i.e., facial expressions and voluntary head motions); (2) the proposed method was verified by comparing with the previous methods using the peak detection and the FFT; (3) the proposed method can be combined with existing vision-based measurement and can improve their performance; (4) the proposed method was tested by three experiments considering the realistic environment including the motion artifacts, thus, it increases the possibility of the non-contact measurement in daily life.