Built-in Camera Research Articles

A Cooking Recipe Recommendation System with Visual Recognition of Food Ingredients

In this paper, we propose a cooking recipe recommendation system which runs on a consumer smartphone as an interactive mobile application. The proposed system employs real-time visual object recognition of food ingredients, and recommends cooking recipes related to the recognized food ingredients. Because of visual recognition, by only pointing a built-in camera on a smartphone to food ingredients, a user can get to know a related cooking recipes instantly. The objective of the proposed system is to assist people who cook to decide a cooking recipe at grocery stores or at a kitchen. In the current implementation, the system can recognize 30 kinds of food ingredient in 0.15 seconds, and it has achieved the 83.93% recognition rate within the top six candidates. By the user study, we confirmed the effectiveness of the proposed system.

Enhanced real-time head pose estimation system for mobile device

This article proposes a real-time Head Pose Estimation (HPE) technique designed to be used in mobile devices. The method enables the interaction between the user and mobile devices using the device's inbuilt camera. The proposed technique is composed

Effect of A Time-lapse Incubator (EmbryoScope®) onin vitroCulture of Human Embryos

A time-lapse embryo image monitoring system consists of an incubator with a built-in microscope and CCD camera. Embryo assessment can be performed with this type of system without removing embryos from the incubator, which eliminates the risks of stress due to temperature changes, light exposure, high oxygen exposure, and pH changes in the culture medium. We cultured embryos to the blastocyst stage using this system and compared the results with those of a normal incubator. We compared the rates of fertilization and cryopreservation of embryos cultured using EmbryoScope® and a conventional incubator. We then compared the pregnancy rates of EmbryoScope® and the conventional incubator. We finally examined the relationship between the early cleavage of embryos and blastocyst formation rate. The fertilization rates (EmbryoScope®: 74.1%, 298/402; conventional incubator: 77.4%, 202/261) and cryopreservation rates (EmbryoScope®: 57.4%, 163/284; conventional incubator: 48.7%o, 91/187) of embryos were not affected by the incubator. No significant difference was observed in clinical pregnancy rates between the groups: (EmbryoScope®: 24.5%, 24/98; conventional incubator: 15.0%o, 15/100). The first division time of embryos was significantly faster in the “2-cell” groups (Groups A and B) (P < 0.01) than in the “none 2-cell” group (Group C). The blastocyst formation rates of embryos in Groups A and B were significantly (P < 0.05) higher than those in Group C. These results indicated that EmbryoScope® enables a detailed and dynamic analysis of the development of human embryos.

Audio/visual concepts for human/robot communication in immersive virtual environments

Communication between robots and humans is a challenge in complex built environments. In this project, we are exploring how previous achievements in this area apply to human/robot communication within immersive virtual displays. In our concrete example, we host human-scale robots that interact with humans in our Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab). The CRAIVE-Lab provides a physical/digital environment for collaborative tasks using seamless video from multiple projectors and a 128-channel wave-field system. The system is designed to monitor the whole floor-space area (10 m × 12 m) with a camera network mounted to the ceiling grid and a microphone array combining shotgun, spherical and possibly people-worn microphones. Based on the sensor data, a visual analysis and an auditory scene analysis are performed. The latter includes sound localization, speech and musical-feature recognition. The robots, a Rethink Baxter robot mounted on an electric wheelchair and a Robokind Zeno, are used to perform assistive technology and social communication tasks. Both robots have direct access to these data using a digital feed over a wireless network to augment their own sensor systems of built-in cameras and microphones. [Work supported by NSF grant #1229391.]

Immunochromatographic diagnostic test analysis using Google Glass.

We demonstrate a Google Glass-based rapid diagnostic test (RDT) reader platform capable of qualitative and quantitative measurements of various lateral flow immunochromatographic assays and similar biomedical diagnostics tests. Using a custom-written Glass application and without any external hardware attachments, one or more RDTs labeled with Quick Response (QR) code identifiers are simultaneously imaged using the built-in camera of the Google Glass that is based on a hands-free and voice-controlled interface and digitally transmitted to a server for digital processing. The acquired JPEG images are automatically processed to locate all the RDTs and, for each RDT, to produce a quantitative diagnostic result, which is returned to the Google Glass (i.e., the user) and also stored on a central server along with the RDT image, QR code, and other related information (e.g., demographic data). The same server also provides a dynamic spatiotemporal map and real-time statistics for uploaded RDT results accessible through Internet browsers. We tested this Google Glass-based diagnostic platform using qualitative (i.e., yes/no) human immunodeficiency virus (HIV) and quantitative prostate-specific antigen (PSA) tests. For the quantitative RDTs, we measured activated tests at various concentrations ranging from 0 to 200 ng/mL for free and total PSA. This wearable RDT reader platform running on Google Glass combines a hands-free sensing and image capture interface with powerful servers running our custom image processing codes, and it can be quite useful for real-time spatiotemporal tracking of various diseases and personal medical conditions, providing a valuable tool for epidemiology and mobile health.

A Mobile Phone User Interface for Image-Based Dietary Assessment.

Many chronic diseases, including obesity and cancer, are related to diet. Such diseases may be prevented and/or successfully treated by accurately monitoring and assessing food and beverage intakes. Existing dietary assessment methods such as the 24-hour dietary recall and the food frequency questionnaire, are burdensome and not generally accurate. In this paper, we present a user interface for a mobile telephone food record that relies on taking images, using the built-in camera, as the primary method of recording. We describe the design and implementation of this user interface while stressing the solutions we devised to meet the requirements imposed by the image analysis process, yet keeping the user interface easy to use.

Detection and Spatial Mapping of Mercury Contamination in Water Samples Using a Smart-Phone

Detection of environmental contamination such as trace-level toxic heavy metal ions mostly relies on bulky and costly analytical instruments. However, a considerable global need exists for portable, rapid, specific, sensitive, and cost-effective detection techniques that can be used in resource-limited and field settings. Here we introduce a smart-phone-based hand-held platform that allows the quantification of mercury(II) ions in water samples with parts per billion (ppb) level of sensitivity. For this task, we created an integrated opto-mechanical attachment to the built-in camera module of a smart-phone to digitally quantify mercury concentration using a plasmonic gold nanoparticle (Au NP) and aptamer based colorimetric transmission assay that is implemented in disposable test tubes. With this smart-phone attachment that weighs <40 g, we quantified mercury(II) ion concentration in water samples by using a two-color ratiometric method employing light-emitting diodes (LEDs) at 523 and 625 nm, where a custom-developed smart application was utilized to process each acquired transmission image on the same phone to achieve a limit of detection of ∼3.5 ppb. Using this smart-phone-based detection platform, we generated a mercury contamination map by measuring water samples at over 50 locations in California (USA), taken from city tap water sources, rivers, lakes, and beaches. With its cost-effective design, field-portability, and wireless data connectivity, this sensitive and specific heavy metal detection platform running on cellphones could be rather useful for distributed sensing, tracking, and sharing of water contamination information as a function of both space and time.

Fundus imaging with a mobile phone: a review of techniques.

Fundus imaging with a fundus camera is an essential part of ophthalmic practice. A mobile phone with its in-built camera and flash can be used to obtain fundus images of reasonable quality. The mobile phone can be used as an indirect ophthalmoscope when coupled with a condensing lens. It can be used as a direct ophthalmoscope after minimal modification, wherein the fundus can be viewed without an intervening lens in young patients with dilated pupils. Employing the ubiquitous mobile phone to obtain fundus images has the potential for mass screening, enables ophthalmologists without a fundus camera to document and share findings, is a tool for telemedicine and is rather inexpensive.

Automatic Number Plate Recognition (ANPR) through smart Phones using Image Processing Techniques

Correct recognition of vehicles violating traffic rules and regulations is a major challenge in the present complex traffic environment. An automatic Number Plate Recognition (ANPR) system plays an important role in the detection of such events. In this paper, automatic number plate recognition (ANPR) was designed and implemented on Android mobile phone platform. First, the graphical user interface (GUI) for capturing image using built-in camera was developed to acquire car plate number. Second, the preprocessing of raw image was done using contrast enhancement, filtering, and straightening. Next, an optical character recognition (OCR) using neural network was utilized to extract texts and numbers. The proposed ANPR algorithm was implemented and simulated using Android SDK on a smart phone or PC. Future research includes optimizing the system for mobile phone implementation with limited CPU and memory resources, and geo-tagging of the image using GPS coordinates and online database for various mobile applications.

Extraction of Character Sequence from Signboards by Using Background Color Information

The smart phones, tablet terminals and mobile phones with built-in cameras are today very popular because of their ease of use. It is important for image processors to be able identify a signboard in an image and extract the character sequence. Techniques for extracting a character sequence from a signboard can combined with various other systems to realize a diverse range of services. When utilization of the system is assumed, it is necessary to consider improvement in the extraction accuracy of a character string domain, and improvement in the speed of the processing. Therefore, this paper proposes an algorithm for extracting character sequence from signboards on the basis of scene image features. In order to improve processing speed and accuracy, the information about the background color of the signboard obtained by cooperation with the car-navigation system etc. was also utilized. In the experiment conducted for 319 color signboards, the proposed method detected 313 signboards (98.12%), and for 569 monochrome signboards, the proposed method detected 513 signboards (90.16%). Moreover, the average processing time of the proposed method was about 0.47 seconds for a color signboard, and was about 0.95 seconds for a monochrome signboard. The experimental results revealed that the proposed method was able to accurately extract the different type of character string on signboard.

Development of an Algorithm for Heart Rate Measurement Using a Mobile Phone Camera

Nowadays there exist many different ways to measure a person’s heart rate. One of them assumes the usage of a mobile phone built-in camera. This method is easy to use and does not require any additional skills or special devices for heart rate measurement. It requires only a mobile cellphone with a built-in camera and a flash. The main idea of the method is to detect changes in finger skin color that occur due to blood pulsation. The measurement process is simple: the user covers the camera lens with a finger and the application on the mobile phone starts catching and analyzing frames from the camera. Heart rate can be calculated by analyzing average red component values of frames taken by the mobile cellphone camera that contain images of an area of the skin. In this paper the authors review the existing algorithms for heart rate measurement with the help of a mobile phone camera and propose their own algorithm which is more efficient than the reviewed algorithms.

Skew Detection based on Bounding Edge Approximation

Any paper document when converted to electronic form through standard digitizing devices, like scanners, is subject to a small tilt or skew. With recent advances of hand-held devices such as cell-phones, Personal Digital Assistants (PDA), etc. having built-in digital cameras, a new trend of document capturing has emerged. Because of the non-contact nature of digital cameras attached to these handheld devices, acquired images very often suffer from skew distortion. A de-skewed document allows a more compact representation of its components, particularly text objects, such as words, lines, and paragraphs. This simplified representation leads to more efficient, robust, as well as simpler algorithms for document image analysis including optical character recognition (OCR). This paper presents a new method for automatic skew detection in images using heuristic approach .The proposed algorithm is fast as well as independent of the scripting language.

About neural-network algorithms application in viseme classification problem with face video in audiovisual speech recognition systems

The paper considers the phoneme recognition by facial expressions of a speaker in voice-activated control systems. We have developed a neural network recognition algorithm by using the phonetic words decoding method and the requirement for isolated syllable pronunciation of voice commands. The paper presents the experimental results of viseme (facial and lip position corresponding to a particular phoneme) classification of Russian vowels. We show the dependence of the classification accuracy on the used classifier (multilayer feed-forward network, support vector machine, k-nearest neighbor method), image features (histogram of oriented gradients, eigenvectors, SURF local descriptors) and the type of camera (built-in or Kinect one). The best accuracy of speaker-dependent recognition is shown to be 85% for a built-in camera and 96% for Kinect depth maps when the classification is performed with the histogram of oriented gradients and the support vector machine.

Slit-lamp handling: Learning upgrade with webcam video recordings

Slit lamp handling has always been difficult for students of the degree of Optics and Optometry in Spain. Instruments with associated cameras help a lot in this task. They allow teachers to observe and control students learning and performance. However, these devices are more expensive than those that do not have an integrated camera connected to a display unit. In this paper we present a cost effective alternative. We propose to place a webcam in one of the slit-lamp oculars (in our case, a Microsoft Lifecam HD-5000). The webcams are connected to a PC running Linux Ubuntu 11.0; therefore that is a low-cost device. The system was used during the teaching of slit-lamp handling in the practice of the subject of Contactologia I of the degree in Optics and Optometry at the Universidade de Santiago de Compostela (Spain). The students were asked to complete a survey in order to test its usefulness. 90% of the students found the system very helpful. Our experience shows that with this simple approach we can do things easier: show the slit-lamp handling to all the students at the same time; take pictures or videos of different eye health conditions or exploratory routines for posterior visualization with all the students; increase the interactions between students allowing them to help and correct each other; and also record the final routine exam in order to make possible its revision with the students. We think that the presented option is a cost effective alternative to built-in cameras provided by slit lamp manufacturers for teaching purposes, supporting the training in optometry practice and increasing the student´s confidence without a big outlay.

Development of Portable Automatic Number Plate Recognition System on Android Mobile Phone

The Automatic Number Plate Recognition (ANPR) System has performed as the main role in various access control and security, such as: tracking of stolen vehicles, traffic violations (speed trap) and parking management system. In this paper, the portable ANPR implemented on android mobile phone is presented. The main challenges in mobile application are including higher coding efficiency, reduced computational complexity, and improved flexibility. Significance efforts are being explored to find suitable and adaptive algorithm for implementation of ANPR on mobile phone. ANPR system for mobile phone need to be optimize due to its limited CPU and memory resources, its ability for geo-tagging image captured using GPS coordinates and its ability to access online database to store the vehicle's information. In this paper, the design of portable ANPR on android mobile phone will be described as follows. First, the graphical user interface (GUI) for capturing image using built-in camera was developed to acquire vehicle plate number in Malaysia. Second, the preprocessing of raw image was done using contrast enhancement. Next, character segmentation using fixed pitch and an optical character recognition (OCR) using neural network were utilized to extract texts and numbers. Both character segmentation and OCR were using Tesseract library from Google Inc. The proposed portable ANPR algorithm was implemented and simulated using Android SDK on a computer. Based on the experimental results, the proposed system can effectively recognize the license plate number at 90.86%. The required processing time to recognize a license plate is only 2 seconds on average. The result is consider good in comparison with the results obtained from previous system that was processed in a desktop PC with the range of result from 91.59% to 98% recognition rate and 0.284 second to 1.5 seconds recognition time.

Real time quantitative colourimetric test for methamphetamine detection using digital and mobile phone technology

The Simon presumptive color test was used in combination with the built-in digital camera on a mobile phone to detect methamphetamine. The real-time Red-Green-Blue (RGB) basic color data was obtained using an application installed on the mobile phone and the relationship profile between RGB intensity, including other calculated values, and the colourimetric product was investigated. A wide linear range (0.1–2.5mgmL−1) and a low detection limit (0.0110±0.0001–0.044±0.002mgmL−1) were achieved. The method also required a small sample size (20μL). The results obtained from the analysis of illicit methamphetamine tablets were comparable to values obtained from gas chromatograph-flame ionization detector (GC-FID) analysis. Method validation indicated good intra- and inter-day precision (2.27–4.49%RSD and 2.65–5.62%RSD, respectively). The results suggest that this is a powerful real-time mobile method with the potential to be applied in field tests.

Respiratory Rate Estimation from the Built-in Cameras of Smartphones and Tablets

This paper presents a method for respiratory rate estimation using the camera of a smartphone, an MP3 player or a tablet. The iPhone 4S, iPad 2, iPod 5, and Galaxy S3 were used to estimate respiratory rates from the pulse signal derived from a finger placed on the camera lens of these devices. Prior to estimation of respiratory rates, we systematically investigated the optimal signal quality of these 4 devices by dividing the video camera's resolution into 12 different pixel regions. We also investigated the optimal signal quality among the red, green and blue color bands for each of these 12 pixel regions for all four devices. It was found that the green color band provided the best signal quality for all 4 devices and that the left half VGA pixel region was found to be the best choice only for iPhone 4S. For the other three devices, smaller 50×50 pixel regions were found to provide better or equally good signal quality than the larger pixel regions. Using the green signal and the optimal pixel regions derived from the four devices, we then investigated the suitability of the smartphones, the iPod 5 and the tablet for respiratory rate estimation using three different computational methods: the autoregressive (AR) model, variable-frequency complex demodulation (VFCDM), and continuous wavelet transform (CWT) approaches. Specifically, these time-varying spectral techniques were used to identify the frequency and amplitude modulations as they contain respiratory rate information. To evaluate the performance of the three computational methods and the pixel regions for the optimal signal quality, data were collected from 10 healthy subjects. It was found that the VFCDM method provided good estimates of breathing rates that were in the normal range (12-24 breaths/min). Both CWT and VFCDM methods provided reasonably good estimates for breathing rates that were higher than 26 breaths/min but their accuracy degraded concomitantly with increased respiratory rates. Overall, the VFCDM method provided the best results for accuracy (smaller median error), consistency (smaller interquartile range of the median value), and computational efficiency (less than 0.5s on 1min of data using a MATLAB implementation) to extract breathing rates that varied from 12 to 36 breaths/min. The AR method provided the least accurate respiratory rate estimation among the three methods. This work illustrates that both heart rates and normal breathing rates can be accurately derived from a video signal obtained from smartphones, an MP3 player and tablets with or without a flashlight.

Small screen and big screen: Mobile film-making in Australasia

Abstract Over the past decade, technological advances have enabled the transformation of mobile phones with built-in cameras into sophisticated digital media tools (smartphones), capable of recording, editing and sharing high definition video content across global communication networks. Advances in smartphone technologies, and in particular increased lens quality, data memory and improved camera functionality, have arguably contributed to the emergence of a new kind of film-maker who recognizes and exploits the creative potential presented by camera phones. In this article, the authors will examine the challenges and perceived opportunities for film-makers using camera phones in an Australasian media context, including the ways in which camera phones may facilitate innovative practices around the production, sharing and viewing of digital films. The subsequent emergence of dedicated festivals for the screening of video content captured on camera phones is driving interest in new forms, new genres and new practices in film-making. We outline ways in which these festivals provide ‘amateur’ and independent film-makers, along with audiences, with a vital platform to initiate discussion, debate and the sharing of video content produced by film-makers from culturally diverse backgrounds.

AUTHENTICATION OF FINGERPRINT BIOMETRICS ACQUIRED USING A CELLPHONE CAMERA: A REVIEW

Over the past decade, there have been dramatic increases in the usage of mobile phones in the world. Currently available smart mobile phones are capable of storing enormous amounts of personal information/data. The smart mobile phone is also capable of connecting to other devices, with the help of different applications. Consequently, with these connections comes the requirement of security to protect personal information. Nowadays, in many applications, a biometric fingerprint recognition system has been embedded as a primary security measure. To enable a biometric fingerprint recognition system in smart mobile phones, without any additional costs, a built-in high performance camera can be utilized. The camera can capture the fingerprint image and generate biometric traits that qualify the biometric fingerprint authentication approach. However, the images acquired by a mobile phone are entirely different from the images obtained by dedicated fingerprint sensors. In this paper, we present the current trend in biometric fingerprint authentication techniques using mobile phones and explore some of the future possibilities in this field.

Construction of Virtual Petrochemical Enterprises Based on Virtual Reality Technology

As petrochemical production equipment with a high degree of risk, on-site training of operating staff and management has brought a great deal of difficulty. Therefore, this paper introduces our research and development work based on virtual reality technology to construct a virtual environment of petrochemical enterprises for on-site training of operating staff and management. To establish the virtual petrochemical enterprises, we adopted 3DMAX as a modeling tool, the VRP (Virtual Reality Platform) as a virtual reality platform as an operation platform. First of all, we create the various sub-scene of the area of petrochemical plants and do model integration and processing to result in the virtual scene with the virtual reality software. Then, we set interactive interface, allowing users to interact with the platform, which makes operations easier. We built-in virtual cameras and pre-recorded screen video to help play the presentation of the video content automatically, so that without human interaction will require. Finally, the role of script will blend with the real three-dimensional scene, the 2D panel and the daemon in order to obtain the true visual display. Therefore, the developed virtual petrochemical enterprise platform has a good show with its real intuitive.