Landmark Features Research Articles

Drowsiness Detection Based on Facial Landmark and Uniform Local Binary Pattern

When driving a vehicle, it is often challenging for someone to force his condition to keep driving even though in sleepy condition, thus causing a traffic accident. One of the characteristics of drowsy drivers is the eyes are closed for a certain period. This research proposes a system to detect drowsiness, thus can alert the drowsy driver. The first step is to detect the face using a Funnel-structured cascade algorithm. And then extract the facial landmark features on the face to get the eyes location. The features of eyes are extracted by using a Uniform Local Binary Pattern (ULBP) and the Eyes Aspect Ratio (EAR). EAR is the distance between points at eye landmarks. After the features have been extracted, the system classifies the eyes, whether closed or open by using Support Vector Machine (SVM) method. The system calculates the percentage of eye closure (PERCLOS) to detect drowsiness. Based on the experimental results, the proposed method yields the best accuracy of 95.5% and the optimal value of PERCLOS in drowsiness detection is greater than or equal to 60% with a period of 20 frames.

이미지 특징점 가중치 학습을 통한 내비게이션

Visual navigation is a challenging subject in robotics, which is involved estimating the target position and direction at an arbitrary location. In this study, we follow the snapshot model, a bio-inspired model to determine the target direction with the snapshots taken at the current location and the target location. From the snapshots, we collect landmarks with three different features, the corner landmarks with SURF (Speeded Up Robust Features), the vertical edge landmarks with HOG (Histogram of Gradient) and the Haar-like feature landmarks. Those methods can play significant roles in finding appropriate visual features depending on the environment. A linear combination of those landmarks, that is, weighted feature landmarks are more suitable to find homing vector than landmarks found with one method alone. We propose that the gradient-descent method should be applied to the weighted feature landmarks to improve the homing performance. The homing results with ALV (Average Landmark Vector) model are demonstrated to show the effectiveness of the method.

2D Pose-Invariant Face Recognition Using Single Frontal-View Face Database

Personal identification systems that use face recognition work well for test images with frontal view face, but often fail when the input face is a pose view. Most face databases come from picture ID sources such as passports or driver’s licenses. In such databases, only the frontal view is available. This paper proposes a method of 2D pose-invariant face recognition that assumes the search database contains only frontal view faces. Given a non-frontal view of a test face, the pose-view angle is first calculated by matching the test image with a database of canonical faces with head rotations to find the best matched image. This database of canonical faces is used only to find the head rotation. The database does not contain images of the test face itself, but has a selection of template faces, each face having rotation images of − 45°, − 30°, − 15°, 0°, 15°, 30°, and 45°. The landmark features in the best matched rotated canonical face such as say rotation 15° and it’s corresponding frontal face of rotation 0° are used to create a warp transformation to convert the 15° rotated test face to a frontal face. This warp will introduce some distortion artifacts since some features of the non-frontal input face are not visible due to self-occlusion. The warped image is, therefore, enhanced by mixing intensities using the left/right facial symmetry assumption. The enhanced synthesized frontal face image is then used to find the best match target in the frontal face database. We test our approach using CMU Multi-PIE database images. Our method performs with acceptable and similar accuracy to conventional methods, while using only frontal faces in the test database.

Testing the unified model of vision and attention: Effects of landmark features, stimulus identity and visual eccentricity on visual orienting and conscious discrimination

ABSTRACTThe unified model of vision and attention (Lambert, A. J., Wilkie, J., Greenwood, A., Ryckman, N., Sciberras-Lim, E., Booker, L.-J., & Tahara-Eckl, L. [2018]. Towards a unified model of vision and attention: Effects of visual landmarks and identity cues on covert and overt attention movements. Journal of Experimental Psychology: Human Perception and Performance, 44, 412–432) was tested by examining effects of centrally and peripherally presented landmark and identity cues on attentional orienting. When presented centrally, both kinds of cue influenced attention; but, when presented peripherally, landmark cues affected attention, but identity cues did not. Effects of peripheral landmark cues on saccadic latencies and manual response times for discriminating between target stimuli were as large as those associated with central landmark cues. Because the dorsal stream represents peripheral vision quite well, whereas the ventral stream does not, these findings are consistent with our proposal that landmark cues tend to recruit dorsal stream encoding, while identity cues rely on ventral stream encoding. When participants discriminated consciously between stimuli employed as cues in the attentional orienting task, centrally presented stimuli were discriminated more rapidly than peripheral stimuli, regardless of cue type. This is consistent with the view that conscious discrimination of the stimuli used as landmark and identity cues in the orienting task, requires vision for perception and recruits ventral stream encoding.

Bimodal Emotion Recognition Using Audio and Facial Features

A multimodal emotion recognition system is proposed using speech and facial images. For this purpose a video database is developed, containing emotions in three affective states viz. anger, sad and happiness. The audio and the snapshots of facial expressions acquired from the videos constituted the bimodal input for recognizing emotions. The spoken sentences in the database included text dependent as well as text independent sentences in Malayalam language. The audio features included short-time processing of speech to obtain: energy, zero crossing count, pitch and Mel Frequency Cepstral Coefficients. For facial expressions, the landmark features of face: eyebrows, eyes and mouth, obtained using Viola Jones Algorithm is used. The supervised learning methods K-Nearest Neighbor and Artificial Neural Network are used for emotion analysis. The system performance is evaluated for 3 cases viz. using audio based features and facial features separately and for both features taken together. Further, the effect of text dependent and text independent audio is also analyzed. The result of the analysis shows that text independent videos (utilizing both modalities) using K-Nearest Neighbor (highest accuracy 82.78%) is found to be more effective in recognizing emotions from the database considered.

Automatic Localization of Landmarks in Craniomaxillofacial CBCT Images Using a Local Attention-Based Graph Convolution Network.

Landmark localization is an important step in quantifying craniomaxillofacial (CMF) deformities and designing treatment plans of reconstructive surgery. However, due to the severity of deformities and defects (partially missing anatomy), it is difficult to automatically and accurately localize a large set of landmarks simultaneously. In this work, we propose two cascaded networks for digitizing 60 anatomical CMF landmarks in cone-beam computed tomography (CBCT) images. The first network is a U-Net that outputs heatmaps for landmark locations and landmark features extracted with a local attention mechanism. The second network is a graph convolution network that takes the features extracted by the first network as input and determines whether each landmark exists via binary classification. We evaluated our approach on 50 sets of CBCT scans of patients with CMF deformities and compared them with state-of-the-art methods. The results indicate that our approach can achieve an average detection error of 1.47mm with a false positive rate of 19%, outperforming related methods.

Maintaining Visibility of a Landmark using Optimal Sampling-based Path Planning

An approach to extend sampling-based path planning algorithms to include visual restrictions is presented. This approach imposes of visual constraints during the sampling and optimization processes. Four visual constraints are imposed during sampling: 1) keep the landmark within the sensor field of view, 2) avoid landmark occlusions, 3) maintaining landmark features near the image center, and 4) limit changes in landmark view orientation. These last two are imposed during path optimization. The robot task is to maintain these constraints, in an environment with obstacles, while the robot changes configurations. The sampling-based motion planning algorithm imposes and maintains both physical and visual restrictions. The process uses a collision checker to detect self-and obstacle-collisions, or landmark occlusions. To infer the landmark visibility, the algorithm dynamically builds a 3D model of camera field visibility as seen from the moving robot. To maintaining the landmark features close to the image center, a distance parameter from the field of view boundary to the landmark is used and optimized. The camera roll angle was included as another element to be optimized, limiting changes in orientation. The algorithm has been implemented, and both results in simulation and experiments using a real robot manipulator are presented.

Automatic segmentation algorithms and personalized geometric modelling for a human knee

Abstract Human knee is one of the most complex joints. Different reasons may lead to knee instability. A personalized mathematical model of the knee may improve both diagnostic procedure and knee surgery outcomes. Such models require accurate geometric representation of bones and attachment sites of ligaments and tendons. This paper addresses automatic segmentation of knee bones and detection of origins and insertions for tendons and ligaments. The approach is based on anatomical features of bones and landmarks of tendons/ligaments attachments on the CT images. It provides a tool for the design of patient-specific geometrical knee models.

VIDEO IMAGE TARGET RECOGNITION AND GEOLOCATION METHOD FOR UAV BASED ON LANDMARKS

Abstract. Relying on landmarks for robust geolocation of drone and targets is one of the most important ways in GPS-denied environments. For small drones，there is no direct orientation capability without high-precision IMU. This paper presents an automated real-time matching and geolocation algorithm between video keyframes and landmark database based on the integration of visual SALM and YOLOv3 deep learning network method. The algorithm mainly extracts the landmarks from the drone video keyframe images to improve target geolocation accuracy, and designs different processing scheme of the keyframes which contains rich and spare landmarks. For feature extraction matching, we improved ORB feature extraction strategy, and obtained a more uniformly distributed feature points than original ORB feature extraction. In the three groups of top-down drone video images experiments, the 100 m, 200 m, and 300 m of the case were carried out to verify the robustness of the algorithm and being compared with GPS surveying data. The results show that the features of keyframe landmarks in the top-down video images within 300 m are stable to match the landmark database, the geolocation accuracy is controlled within 0.8 m, and it has good accuracy.

3D convolution network and Siamese-attention mechanism for expression recognition

Researches on Expression recognition focus on common subject-independent task, while cross-database evaluation is rare and lack of universal protocol. The key challenge for both tasks is to extract features that effectively describe the pattern of expression. In this paper, we present a variable length 3D convolution network that is able to output variable length features. Additionally, we proposed a Siamese 3D convolution network that utilize the“neutral, intermediate, peak” frames from another subject to provide attention weights for the extracted features. Furthermore, we proposed a method to extract fixed length landmark features from expression sequence as auxiliary for convolution network. At last, we try to recommend a universal protocol for cross-database evaluation. Experiments on both subject-independent task and cross-database evaluation show that our network not only achieves comprehensive better performance than previous methods, but also have better generalization ability due to the attention mechanism.

Screening by changes in stereotypical behavior during cell motility

Stereotyped behaviors are series of postures that show very little variability between repeats. They have been used to classify the dynamics of individuals, groups and species without reference to the lower-level mechanisms that drive them. Stereotypes are easily identified in animals due to strong constraints on the number, shape, and relative positions of anatomical features, such as limbs, that may be used as landmarks for posture identification. In contrast, the identification of stereotypes in single cells poses a significant challenge as the cell lacks these landmark features, and finding constraints on cell shape is a non-trivial task. Here, we use the maximum caliber variational method to build a minimal model of cell behavior during migration. Without reference to biochemical details, we are able to make behavioral predictions over timescales of minutes using only changes in cell shape over timescales of seconds. We use drug treatment and genetics to demonstrate that maximum caliber descriptors can discriminate between healthy and aberrant migration, thereby showing potential applications for maximum caliber methods in automated disease screening, for example in the identification of behaviors associated with cancer metastasis.

Coastal SLAM With Marine Radar for USV Operation in GPS-Restricted Situations

The global positioning system (GPS) has become an indispensable navigation sensor for field operations with unmanned surface vehicles (USVs) in marine environments. However, GPS may not always be available, even in open outdoor areas, because it is vulnerable to natural interference and malicious jamming attacks. Thus, an alternative navigation system is required when the use of GPS is restricted or prohibited. In such circumstances, a marine radar, which is a standard sensor in a marine vehicle including USV, can be used for localization in coastal areas. The marine radar can extract landmark features of the surrounding coastlines. These features can be utilized for relative navigation with respect to the detected coastlines. However, coastline maps based on radar signatures may be unavailable in unexplored areas, and they may be unreliable in coastal areas with high tidal elevations. In this study, the relative navigation with respect to the surrounding coastlines is performed in the framework of simultaneous localization and mapping (SLAM) for a USV operation in coastal waters. In particular, coastline features are parameterized by using B-splines for efficient map management, instead of the conventional point cloud representation. To verify and demonstrate the performance of the proposed coastal SLAM algorithm, field experiments were conducted in actual coastal environments. The results are presented and discussed in this paper.

A New Method of Simultaneous Localization and Mapping for Mobile Robots Using Acoustic Landmarks

The simultaneous localization and mapping (SLAM) problem for mobile robots has always been a hotspot in the field of robotics. Simultaneous localization and mapping for robots using visual sensors and laser radar is easily affected by the field of view and ground conditions. According to the problems of traditional sensors applied in SLAM, this paper presents a novel method to perform SLAM using acoustic signals. This method enables robots equipped with sound sources, moving within a working environment and interacting with microphones of interest, to locate itself and map the objects simultaneously. In our case, a method of microphone localization based on a sound source array is proposed, and it was applied as a pre-processing step to the SLAM procedure. A microphone capable of receiving sound signals can be directly used as a feature landmark of a robot observation model without feature extraction. Meanwhile, to eliminate the random error caused by hardware equipment, a sound settled in the middle of two microphones was applied as a calibration sound source to determine the value of the random error. Simulations and realistic experimental results demonstrate the feasibility and effectiveness of the proposed method.

Embryonic development of the endangered Caspian brown trout,Salmo trutta caspius(Kessler, 1877)

The Caspian brown trout, Salmo trutta caspius is an endangered species of economic value and better knowledge of its biology will contribute to conservation measures. In the present study we characterized its early development to identify critical embryonic stages, by observing embryos and larvae from a broodstock maintained in captivity incubated at 9 ± 1°C. Morphological analysis of landmark stages of embryonic development revealed that first cleavage occurred at 189 degree-h, a many-celled blastoderm formed at 62 degree-days (dd), a germ ring at 72 dd, the neural keel stage at 88 dd, the blastopore closed at 137 dd, the eye became pigmented at 188 dd and the larvae hatched at 373 dd. The detailed timing of these landmark features will help development of strategies aiming at increased aquaculture production of this species.

Echolocating bats inspect and discriminate landmark features to guide navigation.

Landmark-guided navigation is a common behavioral strategy for way-finding, yet prior studies have not examined how animals collect sensory information to discriminate landmark features. We investigated this question in animals that rely on active sensing to guide navigation. Four echolocating bats (Eptesicus fuscus) were trained to use an acoustic landmark to find and navigate through a net opening for a food reward. In experimental trials, an object serving as a landmark was placed adjacent to a net opening and an object serving as a distractor was placed next to a barrier (covered opening). The location of the opening, barrier and objects were moved between trials, but the spatial relationships between the landmark and opening, and between the distractor and barrier were maintained. In probe trials, the landmark was placed next to a barrier, while the distractor was placed next to the opening, to test whether the bats relied on the landmark to guide navigation. Vocal and flight behaviors were recorded with an array of ultrasound microphones and high-speed infrared motion-capture cameras. All bats successfully learned to use the landmark to guide navigation through the net opening. Probe trials yielded an increase in both the time to complete the task and the number of net crashes, confirming that the bats relied largely on the landmark to find the net opening. Further, landmark acoustic distinctiveness influenced performance in probe trials and sonar inspection behaviors. Analyses of the animals' vocal behaviors also revealed differences between call features of bats inspecting landmarks compared with distractors, suggesting increased sonar attention to objects used to guide navigation.

FPGA Platform applied for Facial Expression Recognition System using Convolutional Neural Networks

Emotion being a subjective thing, leveraging knowledge and science behind labeled data and extracting the components that constitute it. With the development of deep learning in computer vision, emotion recognition has become a widely-tackled research problem. In this work, we propose a Field Programmable Gate Array (FPGA) architecture applied for this task using independent method called convolutional neural network (CNN). The emotion recognition block receives the detected faces from a video stream by using VITA-2000 camera module and process the image data with the trained CNN model. The architecture is implemented on a Zynq-7000 All Programmable SoC Video and Imaging Kit. Once we have trained a network, weights from the Tensorflow model will be convert as C-arrays, to be used in Vivado HLS. After having the weights as C arrays, they can be implemented to FPGA system. We can also test the functionality of the CNN entirely, by compiling the design with C++ compiler. This method was trained on the posed-emotion dataset (FER2013). The results show that with more fine-tuning and depth, the CNN model can outperform the state-of-the-art methods for emotion recognition. We also propose some exciting ideas for expanding the concept of representational landmark features and sliding windows to improve its performance.

Anatomic Study of Craniocervical Junction and Its Surrounding Structures in Endoscopic Transoral-Transpharyngeal Approach.

This study aims to provide accurate and comprehensive data of craniocervical junction and its peripheral structures in order to provide a profound insight of craniocervical junction as well as to avoid complications during surgical procedures related to it. Computed tomographic angiography (CTA) images of 120 individuals were reviewed, the measurements were performed on coronal, sagittal, and axial planes after 3-dimensional volume reconstruction. The authors measured pharyngeal tubercle, foramen magnum, and tuberculum anterius atlantis, which located based on the position of incisor. The anatomic features of other important bony landmarks, internal carotid artery, and vertebral artery were also fully studied so as to avoid being injured during the transoral-transpharyngeal procedure. During the endoscopic surgery to craniocervical junction, the bending angle of neuroendoscopy should be 14.27 ± 4.51° and the entering depth should be about 72.57 ± 8.72 mm. It is safe to work within the angle of 77.73 ± 3.15° in axial plane and the safe penetration width from the axial midline is 20.05 ± 3.11 mm in the level of foramina magnum. The distance from axial middle line to hypoglossal canal, external opening of carotid canal, and inner edge of jugular foramen was 9.78 ± 0.72, 24.50 ± 1.26, and 24.33 ± 1.68 mm, respectively. These data in this study are valuable for neurosurgeons in clinical practice to reduce the possibility of complications and maximize the safety of surgeries; these data also contribute to the understanding of the anatomy of craniocervical junction and its surrounding structures.

Recognizing gender from human facial regions using genetic algorithm

Recently, recognition of gender from facial images has gained a lot of importance. There exist a handful of research work that focus on feature extraction to obtain gender-specific information from facial images. However, analyzing different facial regions and their fusion help in deciding the gender of a person from facial images. In this paper, we propose a new approach to identify gender from frontal facial images that is robust to background, illumination, intensity, and facial expression. In our framework, first the frontal face image is divided into a number of distinct regions based on facial landmark points that are obtained by the Chehra model proposed by Asthana et al. The model provides 49 facial landmark points covering different regions of the face, e.g., forehead, left eye, right eye, lips. Next, a face image is segmented into facial regions using landmark points and features are extracted from each region. The compass LBP feature, a variant of LBP feature, has been used in our framework to obtain discriminative gender-specific information. Following this, a support vector machine-based classifier has been used to compute the probability scores from each facial region. Finally, the classification scores obtained from individual regions are combined with a genetic algorithm-based learning to improve the overall classification accuracy. The experiments have been performed on popular face image datasets such as Adience, cFERET (color FERET), LFW and two sketch datasets, namely CUFS and CUFSF. Through experiments, we have observed that, the proposed method outperforms existing approaches.

ELF: Extract Landmark Features By Optimizing Topology Maintenance, Redundancy, and Specificity.

Feature selection is the process of selecting a subset of landmark features for model construction when there are many features and a comparatively few samples. The far-reaching development technologies such as biological sequencing at single cell level make feature selection a more challenging work. The difficulty lies in four facts: those features measured are in high dimension and with noise; dropouts make the data much sparse; many features are either redundant or irrelevant; and samples are not well-labeled in the experiments. Here, we propose a new model called ELF (Extract Landmark Features) to address the above challenges. ELF aims to simultaneously maximize topology maintenance to keep the pairwise relationships among samples, minimize feature redundancy to diversify the features, and maximize feature specificity to make every selected feature more representative. This makes ELF a nonlinear combinatorial optimization. To solve this difficult problem, we propose a heuristic algorithm based on greedy strategy. We show ELF's outstanding performance on two single cell RNA-seq datasets. One is the direct reprogramming from mouse embryonic fibroblasts to induced neuron and the other is hepatoblast differentiation. ELF is able to choose only hundreds of landmark genes to maintain the cells' correlativity. Topology maintenance, redundancy removal, and specificity each plays its important role in selecting landmark features and revealing cells' biological functions. In addition, ELF can be generally applied in other scenarios. We demonstrate that ELF can reveal pivotal pixel in writing region and human face in two public image datasets. We believe that ELF is a useful tool to obtain more interpretable results by revealing key features while clustering the samples well.

Dissociating Landmark Stability from Orienting Value Using Functional Magnetic Resonance Imaging.

Retrosplenial cortex (RSC) plays a role in using environmental landmarks to help orientate oneself in space. It has also been consistently implicated in processing landmarks that remain fixed in a permanent location. However, it is not clear whether the RSC represents the permanent landmarks themselves or instead the orienting relevance of these landmarks. In previous functional magnetic resonance imaging (fMRI) studies, these features have been conflated—stable landmarks were always useful for orienting. Here, we dissociated these two key landmark attributes to investigate which one best reflects the function of the RSC. Before scanning, participants learned the features of novel landmarks about which they had no prior knowledge. During fMRI scanning, we found that the RSC was more engaged when people viewed permanent compared with transient landmarks and was not responsive to the orienting relevance of landmarks. Activity in RSC was also related to the amount of landmark permanence information a person had acquired and, as knowledge increased, the more the RSC drove responses in the anterior thalamus while viewing permanent landmarks. In contrast, the angular gyrus and the hippocampus were engaged by the orienting relevance of landmarks, but not their permanence, with the hippocampus also sensitive to the distance between relevant landmarks and target locations. We conclude that the coding of permanent landmarks in RSC may drive processing in regions like anterior thalamus, with possible implications for the efficacy of functions such as navigation.