Mouth Detection Research Articles

Using Facial Features to Detect Driver’s Drowsiness State

Abstract: Drowsiness and fatigue are one of the main causes leading to road accidents. They can be prevented by taking effort to get enough sleep before driving, drink coffee or energy drink, or have a rest when the signs of drowsiness occur. The popular drowsiness detection method uses complex methods, such as EEG and ECG [19]. This method has high accuracy for its measurement but it need to use contact measurement and it has many limitations on driver fatigue and drowsiness monitor. Thus, it is not comfortable to be used in real time driving. This paper proposes a way to detect the drowsiness signs among drivers by measuring the eye closing rate and yawning. We provide a robust and intelligent strategy for detecting driver tiredness in this work to address this growing problem. This method involves installing a camera inside the car to record the driver's facial look. The first phase involves using computer vision algorithms to identify and track the face region in the recorded video sequence. After that, the head is removed and its lateral and frontal assent are examined for indications of driver weariness. The driver's state is finally assessed during the fusion phase, and if drowsiness is found, a warning message and an alert are given to the driver. Our tests provide strong support for the proposed theory. The parameters of the eyes and mouth detection are created within the face image. The video was change into images frames per second. From there, locating the eyes and mouth can be performed. Once the eyes are located, measuring the intensity changes in the eye area determine the eyes are open or closed. The major goal of this research is to create a non-intrusive system that can recognise driver weariness and deliver a prompt warning.

Using the modified Schirmer test for dry mouth assessment: A cross-sectional study.

This study aimed to establish whether the modified Schirmer test could serve as a diagnostic tool for dry mouth, that is, whether it could reliably measure salivary film at selected locations within the oral cavity, and to identify levels of sensitivity/specificity and determine reference values. Therefore, a cross-sectional study (N = 120, mean age 63.5 [SD 13.9] years) was performed. The test was used at five locations (hard palate; buccal mucosa in molar region at 4 mm above occlusal plane; anterior tongue; lower lip; mouth floor), and results were recorded after 1, 2 and 3min. A statistically significant discriminatory ability of the Schirmer test for the unstimulated salivary flow rates could be shown for the palate (at 3min), buccal mucosa (at 1min), mouth floor (at 1 min), and tongue (at 2 and 3min) (areas under the curve 0.64-0.68), with individual sensitivity/specificity values depending on test location/time points. Thus, the modified Schirmer test has potential to become a simple and reproducible instrument for the detection of dry mouth based on low unstimulated salivary flow rates in dentistry and especially outreach care. Care must be taken concerning intraoral test location and measurement time.

A Pipeline-based Low Complexity Dual Thresholds and Multiresolution Mouth Detection Method

Mouth detection is a basic step in the automatic diagnosis of buccal diseases. This paper proposes a method of using FPGA(Field Prog ram mable Gate Array) to detect the human mouth region in images, which adopts a pipelined design for high-speed operating. Thus, it can be applied to mouth motion detection as well. After an original image is processed regularly by median filtering, color conversion is performed to obtain its corresponding H component, which is used to create a partitioned binary image. Each constituent region of the binary image uses two thresholds for regional processing separately, which results in two binary images: one comprises the mouth and some other regions with a color close to that of the mouth, and the other indicates the position of the mouth. A logical AND is performed on these two binary images to calculate the border of mouth region with reduced resolution, which is then superimposed on the buffered original image to confirm the mouth region eventually. An FPGA-based scheme is presented, which shows the method could detect the whole mouth properly, with a maximum clock speed of 150MHz. In addition, processing an image of 200*200 only consumes 2178 Slice LUTs(Look Up Table), which is a relatively low resource consumption.

Implementation of wheelchair controller using mouth and tongue gesture

In this paper, a simple human-machine interface allowing people with severe disabilities to control a motorized wheelchair using mouth and tongue gesture is presented. The development of the proposed system consists of three principal phases: the first phase is mouth detection which performed by using haar cascade to detect the face area and template matching to detect mouth and tongue gestures from the lower face region. The second phase is command extraction; it is carried by determining the mouth and tongue gesture commands according to the detected gesture, the time taken to execute the gestures, and the previous command which is stored in each frame. Finally, the gesture commands are sent to the wheelchair as instruction using the Bluetooth serial port. The hardware used for this project were; laptop with universal serial bus (USB) webcam as a vision-based control unit, Bluetooth module to receive instructions comes from the vision control unit, standard joystick used in case of emergency, joystick emulator which delivers to the control board signals similar to the signals that are usually generated by the standard joystick, and ultrasonic sensors to provide safe navigation. The experimental results showed the success of the proposed control system based on mouth and tongue gestures.

Audiovisual speaker indexing for Web-TV automations

The current paper introduces a multimodal framework to provide Web-TV automations for live broadcasting and overall big streaming data management. The term indexing refers to the spatiotemporal localization of speakers participating in a discussion panel. Multiple modalities acting in parallel form the data-driven decision-making pipeline. The automated workflow includes the tasks of active speaker detection and localization, frame selection, and creation of a semantically annotated database. For improved performance and robustness, an information fusion model is proposed, which makes use of different audio and visual modalities. Audio-driven Voice Activity Detection follows the Enhanced Temporal Integration methodology applied on a standard audio feature set. For the localization of the dominant audio source, the argument that maximizes the General Cross-Correlation method is calculated. The visual modalities include face and mouth detection and Visual Voice Activity Detection. A Long Short Term Memory network is trained with mouth image sequences to determine voice activity. The values of the audio and visual Voice Activity Detection modules, as well as the General Cross-Correlation result, are used to train an Adaptive Neuro-Fuzzy model, which is responsible for the final decision. Experimental results prove the superiority of the information fusion approach compared to unimodal audio or visual models.

Interventional treatment of Anal Fistula: A Retrospective Analysis

Introduction: Anal fistula is a benign anorectal disease. Infection of Hermann and Desfosses anal glands is responsible for the formation of abscesses and/or fistulas. The main treatment is surgery but recurrence and sometimes incontinence are frequently observed. So to choose surgical methods of anal fistula treatment is very important. Aim: The aim of this retrospective study was to evaluate the reliability and feasibility of Video-assisted anal fistula treatment (VAAFT) at the anal fistula surgery. Materials and Methods: A retrospective analysis was performed on data collected during a 11-year period (2008-2019) from 52 patients who underwent fistulotomy with loose seton technique. The male to female ratio was 3:1 and the mean patient age was 48.34 years. Results: Mortality, recurrence, complications rate were 0%. Though the patients were scheduled as day-surgery cases, the average duration of hospital stay was 1.18 days. Quality of life index (QOLI) for these patients were often be improved by treating their fistula. Conclusion: The appropriate treatment for anal fistula is dependent upon the anatomy and the location of the fistula tract. Detection of the inner mouth is the touchstone in the treatment of anal fistula. It could be determined location of inner mouth of the anal fistula easily with transparent anoscope guideline. VAAFT seems to be a safe and effective technique for treating perianal fistula without recurrence and anal incontinence.

A Comparative Analysis of using Various Machine learning Techniques based on Drowsy Driver Detection

In image processing or computer vision, image segmentation is a vital issue for applications such as scene understanding, medical image evaluation, robotic perception, video surveillance, increased reality or compression, etc. Every year in road accidents caused because of human mistake, the numbers of dead and injured are rising. Drowsiness and driving are particularly risky and difficult to recognize. The second leading cause of road crashes in drowsiness after alcohol. Detecting driver drowsiness is a technology of safety for vehicles that helps placed an end to driver injuries that are dozy. One of the main causes of road accidents is driver drowsiness. It is a very serious issue for road safety. We have presented various methods for detecting the drowsiness of the driverin this paper and the comparisons among such methods are extremely challenging. For this purpose, we have compared machine learning methods based on facial expression, especially on eye state. Apart from eye detection, it performed experiments on mouth detection and face detection as well. This paper explores several methods for machine learning, like SVM, CNN, or HMM. From the analysis, we have found that the HMM model achieved more accurate results in comparisonto others.

Hierarchical deep neural networks to detect driver drowsiness

Driver drowsiness is one of the main reasons for deadly accidents, especially on suburban roads. Researchers have used many methods for analyzing videos and detecting drowsiness, and the most up-to-date methods among them are using deep learning. This paper proposes a hierarchical framework comprising deep networks with split spatial and temporal phases referred to as hierarchical deep drowsiness detection (HDDD) network. The proposed method uses ResNet to detect the driver’s face, lighting condition, and whether the driver is wearing glasses or not. This phase also causes a significant increase in eyes and mouth detection percentage in the next stage. Afterward, the LSTM network is used to take advantage of temporal information between the frames. The average accuracy of the drowsiness detection system is reached 87.19 percent.

Design and Implementation of a Face Recognition System Based on API mobile vision and Normalized Features of Still Images

Biometric identification is the automated technique of measuring the biological data. The term biometrics is commonly used today to recognize a person by analyzing his/her physical characteristics and comparing these characteristics to a database such as fingerprint recognition, retinal scanning, face recognition etc. Face recognition is one of the biometric techniques that are used for identifying and verifying the identity of a person. This paper presents the design and implementation of a face recognition system for Android mobile phone platform. The design methodology includes two main steps. The first step is the extraction of the image’s features and the second one is the recognition according to the classification of patterns. This system includes face detection using face detector available in the Android called object detection (Google’s API mobile vision), features extraction (nose detection, mouth detection, eyes detection and cheek detection) using the same algorithm object detection API. From the feature extraction, we have 30 geometrical measurements of each person. The recognition step is done by calculating the Pearson Correlation Coefficients between the test image’s geometrical measurements and measurements stored in the training database. The system will accept a facial image of the person as authenticated and allow to access the mobile applications if the percentage of correlation is greater than a chosen threshold otherwise it rejects the image. The recognition rate of the proposed approach has achieved an accuracy more than 95% of other approaches.

Semantic Gap Reduction From Mouth Feature Threshold Value Using Viola Jones Algorithm

Image features are useful in defining image precisely and uniquely which are helpful in classification and recognition of images. Various feature values can be extracted by using different techniques used in image processing applications such as pattern recognition, feature matching, image segmentation, image fusion, video processing, visual surveillance, remote sensing, traffic safety monitoring, medical diagnosis, and human computer interaction etc. Number of factors of image affects on accuracy of feature extraction from different databases of images. The distance of the object (or person) from the camera and threshold value for feature detected plays an important role to capture accurate feature from the image database. The runtime required to capture the feature from image is also considerable factor when an algorithm for feature extraction is selected. The paper considers the distance and threshold value used for mouth feature detection and run-time required to capture the mouth feature from face database using Viola Jones algorithm. The experimental result shows increased threshold value at certain level for mouth feature detection gives better result and improved mouth detection rate for semantic gap reduction.

Real-time and efficient eyes and mouth state detection: an artificial intelligence application based on embedded systems

It is reported that many traffic accidents are caused by fatigued driving. Detection of the state of the eyes and mouth is usually employed to judge whether the driver is fatigued. However, the traditional image processing methods cannot achieve satisfactory detection accuracy due to the changes of illumination, head posture, and other factors in the actual environment. To date, although the methods based on deep learning have reached adequate accuracy in the tasks of object detection, they are obliged to rely on high hardware configuration to meet the real-time requirements. To achieve satisfactory accuracy in the task of eyes and mouth detection and obtain good real-time performance on embedded platforms such as NVIDIA Jetson TX2, we improve the object detection algorithm based on deep learning. In this paper, based on the original YOLOv3-Tiny structure, we not only added the structure of deep residual learning, but also calculated six new anchor boxes according to the training set using the K-means algorithm. We also used six data augmentation methods in the training set to improve the detection accuracy. The improved algorithm proposed in this paper can achieve satisfactory detection speed and accuracy on the TX2 embedded hardware configuration platform, validating that the ameliorated scheme is effective.

Hybrid Force Tracking Impedance Control-Based Autonomous Robotic System for Tooth Brushing Assistance of Disabled People

Due to the increase of the world ageing population and several forms of disabilities, assistive robots have received a lot of attention lately. Although, different assistive robotic systems have been developed to perform essential tasks around the head, these existing robotic systems are either manually controlled, shared controlled or interactive ones. Although, fully autonomous robotic systems can have the potential of eliminating several limitations of the existing systems, such systems are limited in design due to critical issues of increased complexities and the associated risks. In this article, a hybrid impedance controller-based autonomous assistive robotic system for tooth brushing assistance is developed. A fixed base manipulator with an attached automatic tooth brush and Kinect RGB-D sensor are proposed as a platform to achieve the brushing task. An online face detection, mouth detection, head pose estimation and trajectory planning for the brushing task are achieved. Additionally, online re-planning the trajectories in case of patients’ head pose changes is taken into account. The controller performance is evaluated regarding tracking the generated brushing trajectories and the desired force trajectory by co-simulating the manipulator’s dynamics in ADAMS and SIMULINK. The results demonstrate the capability of the proposed robotic system to achieve tooth brushing autonomously.

Enhancing Mouth-Based Emotion Recognition Using Transfer Learning.

This work concludes the first study on mouth-based emotion recognition while adopting a transfer learning approach. Transfer learning results are paramount for mouth-based emotion emotion recognition, because few datasets are available, and most of them include emotional expressions simulated by actors, instead of adopting real-world categorisation. Using transfer learning, we can use fewer training data than training a whole network from scratch, and thus more efficiently fine-tune the network with emotional data and improve the convolutional neural network’s performance accuracy in the desired domain. The proposed approach aims at improving emotion recognition dynamically, taking into account not only new scenarios but also modified situations to the initial training phase, because the image of the mouth can be available even when the whole face is visible only in an unfavourable perspective. Typical applications include automated supervision of bedridden critical patients in a healthcare management environment, and portable applications supporting disabled users having difficulties in seeing or recognising facial emotions. This achievement takes advantage of previous preliminary works on mouth-based emotion recognition using deep-learning, and has the further benefit of having been tested and compared to a set of other networks using an extensive dataset for face-based emotion recognition, well known in the literature. The accuracy of mouth-based emotion recognition was also compared to the corresponding full-face emotion recognition; we found that the loss in accuracy is mostly compensated by consistent performance in the visual emotion recognition domain. We can, therefore, state that our method proves the importance of mouth detection in the complex process of emotion recognition.

Real-time video based emotion recognition using convolutional neural network and transfer learning

Background/Objectives: The deep learning approaches have paved their way to construct various artificial intelligence products and the proposed system uses a convolutional neural network for detecting real-time emotions of mankind. The objective of the study is to develop a real-time application for emotion recognition using convolutional neural network and transfer learning methods. Methods/Statistical analysis: The proposed system considers happy, normal and surprised categories of emotions. The system consists of four major steps: dataset collection, training, validation, and real-time testing. The dataset is comprised of face images containing emotions such as happy,normal and surprised in the form of video frames. The face and mouth regions are detected using the Haar-Based Cascade classifier at 20 frames per second. Findings: The convolutional neural network (CNN) is trained using mouth images and the pre-trained models VGG16 and VGG19 are trained with face images. The trained model is used to detect the emotions in the live webcam video. The experimental results show that the CNN model trained using mouth images gives an accuracy of 85.71% and the pre-trained models trained with face images using transfer learning method achieves an accuracy of 77.78%. The proposed system using CNN outperforms the pre-trained models for recognizing the emotions in real-time video. Novelty/Applications:The proposed system is entirely based on the mouth region video frames and the real-time emotion recognition system is developed. This work can detect the three emotions in an unconstrained laboratory environment. Keywords: Convolutional neural network; mouth detection; pre-trained models; real-time emotion recognition; transfer learning

Face Detection Based On Eye-Mouth Triangular Approach

Human face is a complex and dynamic structure. It is a challenge to be able to make a face recognition system like humans. At the beginning of its development, many facial recognition studies only focused on facial features. In 1991, Turk and Pentland developed a face recognition system based on Principal Component Analysis named eigenface. This system is very efficient because it only focuses on components that most affect facial image. However, this system has weaknesses, which cannot be used to determine the position of the face. In this final project, image processing methods will be carried out to detect faces in digital images. The method used is eye mouth triangular approach with the steps being taken are skin detection, eye detection, mouth detection, and facial confirmation. From the results of a hundred digital color images tested, there were 82 images that were successfully detected. The main system failure is caused by failure in skin detection. Further development is needed so that the system can work optimally.

Real-time and efficient eyes and mouth state detection: an artificial intelligence application based on embedded systems

Real-time and efficient eyes and mouth state detection: an artificial intelligence application based on embedded systems

Mouth Gesture Interface for VLC Media Player

This proposed work presents a frame work of mouth gesture recognition for Human Computer Interface (HCI). It replaces the traditional input devices such as mouse and keyboard which allows a user to work on a computer using his/her mouth gestures. This work is aimed at helping severely disabled and paralyzed people. The entire work includes mouth detection, region extraction, gesture classification, and interface creation with computer applications. Initially face and mouth regions are detected using Haar-cascaded classifier. Secondly, the gesture recognition is done using the concept of Deep learning through Convolutional Neural Network (CNN). The mouth gestures are recognized and classified as mouth close, mouth open, tongue left and tongue right. Finally an HCI is created by mapping the mouth gestures into VLC player operations such as play, pause, forward jump and backward jump. The performance of the proposed method is measured and compared with other existing methods. This work is found to perform better than the other methods.

Masked Face Detection using the Viola Jones Algorithm: A Progressive Approach for less Time Consumption

<p>The use of CCTV surveillance is today’s need in<br />public and private sector for ensuring security against terrorism<br />and robbery. Regular expressions are used to signify enormous<br />sets of motion attributes captured in video. The video vigilance<br />is popular system without using human interference to capture<br />important scenes. The motive of the work is to introduce automatic<br />revelation of masked objects in real time with a surveillance<br />camera. The main aim is to detect masked person automatically<br />in less time period. In this paper,the researcher proposes a system<br />that consists methods which uses four variant steps that are the<br />steps of calculating distance range of person from the camera,<br />eye or vision line detection and face part detection such as<br />mouth detection and face detection. Performance of proposed<br />algorithm is carried out on various real time inputs. Experimental<br />evaluation shows that proposed algorithm exceeds better in terms<br />of time consumption. This unique approach for the problem<br />has created a method transparent and easier in complexity so<br />that the real time implementation can be made beneficial and<br />workable. Analysis of the algorithms fulfillment on the test video<br />track gives appropriate judgments for additional improvements<br />in the masked face detection performance. Finally, based on the<br />research, the axioms were useful for the work which can be<br />usually accessible from available algorithms.</p>

Non-intrusive Detection and Prediction of Driver’s Fatigue Using Optimized Yawning Technique

There are hoards evidences that driver fatigue can lead to vehicle accidents so pre alert is must require for such kind of delinquent. This paper taxonomy gives an algorithm for detection of yawing state of a driver driving a vehicle .Whether it is open or closed. The method presented is based upon the non-intrusive method as the camera is used for capturing the image of the driver. A new method is proposed which gives a fast response about yawing. The algorithm presented works upon image processing concept. Owing to its efficiency the same can be implemented in real-time. The results are analyzed and the efficiency obtained for open mouth detection is 98%.

Evaluation of Xerostomia and salivary flow rate in Hashimoto's Thyroiditis.

BackgroundOne of the most common causes of hypothyroidism is Hashimoto´s Thyroiditis (HT). Early detection of dry mouth is critical in preserving and promoting systemic and oral health. In this study we have assessed, for the first time, salivary function and xerostomia in HT patients who have not been involved with Sjögren´s syndrome.Material and MethodsHT was diagnosed in 40 patients based on clinical findings and positive anti-thyroid peroxidase antibodies (anti-TPO). Controls, matched by sex, age and body mass index (BMI), and with no history of thyroid disease, were selected. A questionnaire was used for diagnosis of xerostomia. Saliva samples were taken between 8 a.m. and 9 a.m., and at least 2 hours after the last intake of food or drink. The flow rate was calculated in milliliters per minute.ResultsXerostomia was significantly higher in patients with HT. Unstimulated salivary flow rate was significantly lower in the HT group. Stimulated salivary flow rate was lower in HT group, but the difference was not significant.ConclusionsThe patients with HT experienced xerostomia, and their salivary flow rate was diminished. Spitting the saliva then assessing salivary flow rate based on milliliter per minute is non-invasive, fast, and simple for chair-side diagnosis of dry mouth. Autoimmune diseases can be accompanied by salivary gland dysfunction. This may be due to the effect of cytokines in the autoimmune process or because of thyroid hormone dysfunctions. Key words:Thyroid, salivary gland, xerostomia.