Built-in Camera Research Articles

A Prospective Study of Mobile Phones for Dermatology in a Clinical Setting

We evaluated the accuracy of diagnoses made from pictures taken with the built-in cameras of mobile phones in a 'real-life' clinical setting. A total of 263 patients took part, who photographed their own lesions where possible, and provided clinical information via a questionnaire. After the teledermatology procedure, each patient was examined face-to-face and a gold standard diagnosis was made. The telemedicine data and pictures were diagnosed by 15 dermatologists. The 299 cases contained 1-22 clinical images each (median 3). Nine dermatologists finished all the cases and the remaining six completed some of them, thus providing 2893 decisions. Overall, 61% of all cases were rated as possible to diagnose and of those, 80% were correct in comparison with the face-to-face diagnosis. Image quality was evaluated and the median was 5 on a 10-point scale. There was a significant correlation between the correct diagnosis and the quality of the photographs taken (P < 0.001). In nearly two-thirds of all cases, a teledermatology diagnosis was possible; however, there was insufficient information to make a telemedicine diagnosis in about one-third of the cases. If applied carefully, mobile phones could be a powerful tool for people to optimize their health care status.

Remotely reconfigurable hardware–software platform with web service interface for automated video surveillance

This paper presents a reconfigurable hardware platform for general-purpose video processing. The proposed design is a step towards portable, web-enabled devices which, unlike most existing smart cameras, are to a large degree autonomous and have substantial intelligence embedded. The device is based on programmable logic, microcontrollers, and dedicated communication modules. It is able to acquire an input frame using a built-in camera, process the image in a massively parallel manner, and expose its functionality as a web service compliant with the Service Oriented Architecture (SOA) paradigm. Thanks to the FPGA technology used for main functionality implementation, the system's hardware can be locally or remotely reconfigured in order to optimize it on a very low level for user-defined tasks. Moreover, the image processing core is implemented using a C-to-HDL compiler to reduce time to market when deploying new functionalities. We demonstrate the capabilities of our device on two example applications: moving object detection and face recognition.

From COWs to BYOD

From COWs to BYOD

Mobile teledermoscopy—there’s an app for that!

Background:The introduction of the smartphone with high-quality, built-in digital cameras and easy-to-install software may make it more convenient to perform teledermatology. In this study we looked at the feasibility of using a smartphone (iPhone 4®) with an installed application especially developed for teledermatology (iDoc24®) and a dermoscope (FotoFinder Handyscope®) that is customized to attach to the smartphone to be able to carry out mobile teledermoscopy.Objectives:To study the diagnostic accuracy of this mobile teledermoscopy solution, to determine the interobserver concordance between teledermoscopists (TDs) and a dermatologist meeting the patient face-to-face (FTF), and to assess the adequacy of the TDs’ management decisions and to evaluate the image quality obtained.Patients/Methods:During a 16-week period, patients with one or more suspicious skin lesions deemed to need a biopsy or excision were included. The smartphone app was used to send a clinical image, a dermoscopy image and relevant clinical information to a secure Internet platform (Tele-Dermis®). Two TDs assessed the incoming cases, providing a specific primary diagnosis and a management decision. They also graded the image quality. The histopathological diagnosis was used as the gold standard.Results:Sixty-nine lesions were included. The FTF dermatologist’s diagnostic accuracy was 66.7%, which was statistically higher than TD 1 (50.7%, P=0.04) but similar to TD 2 (60.9%, P=0.52). The interobserver concordances between the FTF dermatologist and the two TDs and between the respective TDs showed moderate to substantial agreement. The TDs provided adequate management decisions for 68 (98.6%) and 69 (100%) lesions, respectively. The image quality was rated as excellent or sufficient in 94% and 84% of the cases by the respective TDs.Conclusion:This novel mobile teledermoscopy solution may be useful as a triage tool for patients referred to dermatologists for suspicious skin lesions.

Using the iPhone as a device for a rapid quantitative analysis of trinitrotoluene in soil

Mobile ‘smart’ phones have become almost ubiquitous in society and are typically equipped with a high-resolution digital camera which can be used to produce an image very conveniently. In this study, the built-in digital camera of a smart phone (iPhone) was used to capture the results from a rapid quantitative colorimetric test for trinitrotoluene (TNT) in soil. The results were compared to those from a digital single-lens reflex (DSLR) camera. The colored product from the selective test for TNT was quantified using an innovative application of photography where the relationships between the Red Green Blue (RGB) values and the concentrations of colorimetric product were exploited. The iPhone showed itself to be capable of being used more conveniently than the DSLR while providing similar analytical results with increased sensitivity. The wide linear range and low detection limits achieved were comparable with those from spectrophotometric quantification methods. Low relative errors in the range of 0.4 to 6.3% were achieved in the analysis of control samples and 0.4–6.2% for spiked soil extracts with good precision (2.09–7.43% RSD) for the analysis over 4 days. The results demonstrate that the iPhone provides the potential to be used as an ideal novel platform for the development of a rapid on site semi quantitative field test for the analysis of explosives.

Using an iPad in research with children and adolescents

The features of the iPad and its widespread adoption support its use as a tool in research involving children and adolescents. Although use in our current research has been limited to a hospital-based setting,the iPad’s design and features support its potential use in home- and community-based settings as well.In addition to collecting data through web-based platforms and applications, the iPad may facilitate telehealth-based interventions through the use of its built-in camera and Face Time applications.In our experience with children and adolescents,the iPad has been a useful tool to support multiple types of data collection as well as support other aspects of the research process. The use of iPad application technology certainly opens up opportunities for new and novel ways to collect meaningful data and support research with individuals across the lifespan.

Handheld Mobile Device Based Text Region Extraction and Binarization of Image Embedded Text Documents

Abstract.Effective text region extraction and binarization of image embedded text documents on mobile devices having limited computational resources is an open research problem. In this paper, we present one such technique for preprocessing images captured with built-in cameras of handheld devices with an aim of developing an efficient Business Card Reader. At first, the card image is processed for isolating foreground components. These foreground components are classified as either text or non-text using different feature descriptors of texts and images. The non-text components are removed and the textual ones are binarized with a fast adaptive algorithm. Specifically, we propose new techniques (targeted to mobile devices) for (i) foreground component isolation, (ii) text extraction and (iii) binarization of text regions from camera captured business card images. Experiments with business card images of various resolutions show that the present technique yields better accuracy and involves low computational overhead in comparison with the state-of-the-art. We achieve optimum text/non-text separation performance with images of resolution 800×600 pixels with an average recall rate of 93.90% and a precision rate of 96.84%. It involves a peak memory consumption of 0.68 MB and processing time of 0.102 seconds on a moderately powerful notebook, and 4 seconds of processing time on a PDA.

IHand: an interactive bare-hand-based augmented reality interface on commercial mobile phones

The performance of mobile phones has rapidly improved, and they are emerging as a powerful platform. In many vision-based applications, human hands play a key role in natural interaction. However, relatively little attention has been paid to the interaction between human hands and the mobile phone. Thus, we propose a vision- and hand gesture-based interface in which the user holds a mobile phone in one hand but sees the other hand’s palm through a built-in camera. The virtual contents are faithfully rendered on the user’s palm through palm pose estimation, and reaction with hand and finger movements is achieved that is recognized by hand shape recognition. Since the proposed interface is based on hand gestures familiar to humans and does not require any additional sensors or markers, the user can freely interact with virtual contents anytime and anywhere without any training. We demonstrate that the proposed interface works at over 15 fps on a commercial mobile phone with a 1.2-GHz dual core processor and 1 GB RAM.

Visual multi-touch air interface for barehanded users by skeleton models of hand regions

In this paper, we propose a visual multi-touch air interface. The proposed air interface is based on the image processing and does not require any additional equipment except a built-in camera. The implemented device provides a barehanded interface which copes with the multi-touch operation. The proposed device is easy to apply to the real-time systems because of its low computational load and is cheaper than the existing methods using glove data or 3-dimensional data because any additional equipment is not required. To improve the robustness of the extraction of hand regions under various circumstances, we propose an image processing algorithm based on the HCbCr color model, the fuzzy color filter, and the labeling. In addition, to improve the accuracy of the recognition of hand gestures, we propose a motion recognition algorithm based on the geometric feature points, the skeleton model, and the Kalman filter. Finally, the experiments show that the proposed device is applicable to remote controllers for video games, smart TVs and any computer applications.

A New Algorithm of Fully-automatic Red-eye Removal

The redeye effect is typically formed in consumer photos taken with a built-in camera flash. The paper presents a new algorithm of the automatic removal of red-eye from digital photos. The proposed algorithm can remove red-eye automatically without manual intervention. First, Detecting faces by using the AdaBoost algorithm. Second, the red-eyes are located by using operations of segmentation, morphology and geometric constraint. Finally, correcting the located red-eyes completely. The experimental results are satisfied with high correction rates, relatively low computational complexity and robustness.

Similar morphokinetic patterns in embryos derived from obese and normoweight infertile women: a time-lapse study

Does female obesity affect the dynamic parameters of embryo quality assessed by time-lapse analysis? Female obesity does not affect the dynamic embryo quality as determined by image acquisition and time-lapse analysis. Female obesity impairs natural and assisted reproduction but there is no agreement on the specific contribution of gametes, embryos or endometrial receptivity. In this preliminary study the dynamic parameters of embryo quality are assessed for the first time by time-lapse analysis. Two-year cohort retrospective study comparing embryos from three groups of patients according to the presence of infertility and/or obesity. Participants attended a University-affiliated private clinic where ICSI was performed. Using an IVF incubator with a built-in camera designed to automatically acquire images at defined time points, we monitored individual embryos from 89 patients: 71 embryos from 13 obese infertile women, 242 embryos from 45 normoweight infertile women and 111 embryos from 31 normoweight fertile oocyte donors. The chronological pattern of cell divisions (timings of cell cleavages) and other morphologic features (time-dependent cell size and nucleation) was recorded. Embryos from obese and normoweight infertile women showed similar cleavage patterns, but occurring more slowly, to those from fertile donors. These differences were statistically significant for t2 (time of cleavage to two-blastomere embryo) (P = 0.016), t3 (P = 0.014), t4 (P = 0.003) and t5 (P = 0.040). These are preliminary data from a retrospective analysis with a limited sample size. Not recommended until further studies using time-lapse analysis of a larger sample have been performed. None.

A Novel Interaction Method for Mobile Devices Using Low Complexity Global Motion Estimation

A novel interaction method for mobile phones using their built-in cameras is presented. By estimating the path connecting the center points of frames captured by the camera phone, objects of interest can be easily extracted and recognized. To estimate the movement of the mobile phone, corners and corresponding Speeded-Up Robust Features descriptors are used to calculate the spatial transformation parameters between the previous and current frames. These parameters are then used to recalculate the locations of the center points in the previous frame into the current frame. The experiment results obtained from real image sequences show that the proposed system is efficient, flexible, and able to provide accurate and stable results.

Higher good blastocyst rate by using an incubator with a built-in a time-lapse embryo image monitoring system

A time-lapse embryo image monitoring system is an incubator with a built-in microscope and CCD camera. Since the embryo assessment can be done without taking out the embryo outside the incubator, it is possible to eliminate the risk of the stress by temperature change, pH change of culture medium and light exposure. We cultured embryos to the blastocyst using this system and compared the acquisition rate of good blastocyst with the rate of good blastocyst cultured in the same period by the conventional incubator.

The effect of passive immunization against ghrelin on feed and water intake in turkeys

Five-week-old turkeys were used to evaluate the effect of passive immunization against ghrelin on feed and water intake and animal behavior. In experiment 1, females were reared using normal feeding and lighting management recommended by the industry. At 5 wk of age (d 0 of experiment 1), birds (n = 40) were individually caged (0.65 × 0.4 × 0.4 m) with free access to feed and water. Feed and water intake were measured 3 times a day (0800, 1200, and 1700 h) by recording the weight of feed or water offered minus any unconsumed feed or water remaining. After 3 d of adaptation to the cages (d 3), birds were stratified by BW and feed consumption and randomly assigned to a 2 × 5 factorial arrangement of treatment. Starting on d 3, turkeys were given intravenous (iv) injections (0.5, 1.0, 2.0, 4.0, or 8.0 mL) of pooled undiluted plasma obtained from pigs that were previously actively immunized against ghrelin or iv injections (0.5, 1.0, 2.0, 4.0, or 8.0 mL) of pooled undiluted plasma, obtained from nonimmunized pigs (control). In experiment 2, the 2 highest doses (i.e., 4.0 and 8.0 mL; n = 4/treatment) were repeated in a 2 × 2 factorial arrangement as described in experiment 1. A laptop computer with a built-in color camera and appropriate software was used to record birds for 9 consecutive hours, starting 4 h before treatments were applied. Video clips were saved and a human observer watched and annotated bird behavior associated with feeding, drinking, and standing. Passively immunized birds increased feed consumption (P = 0.04) compared with control animals. Water intake was not affected by treatments. There was a tendency for immunized birds to increase the number of pecks per hour and the amount of time devoted for feeding. Our data suggest that in turkeys, the effect of immunization against ghrelin on feed intake is the opposite of that observed in mammalian species.

An automated ash fusion test for characterisation of the behaviour of ashes from biomass and coal at elevated temperatures

The benefits of blending biomass with coal for power generation include less CO2 emissions and a reduced dependency on non-renewable fossil fuels. However, there is a need to understand the role of biomass during direct combustion and co-firing, particularly in terms of the effect of biomass use on ash slagging and fouling. A new image analysis based technique has been developed to characterise the behaviour of ashes from biomass, coal and coal/biomass blends using a single heating test at elevated temperatures. It is a reproducible test that combines the conventional ash fusion test, dilatometry and sinter strength test by means of image analysis. An oven is used to heat the cylindrical ash pellets from room temperature to 1520°C, while the in-built camera captures still images of the samples throughout the temperature range. An automated image analysis code has been developed to provide behaviour profiles for each ash sample (across the temperature range) by quantifying dimensional changes upon heating. The error for the determined ash characteristic temperatures is approximately 15°C, which is approximately 50% lower than for a conventional ash fusion test. Cylindrical ash samples from 9 biomasses (corn stover, DDG, DDGS, miscanthus, olive residue, wheat shorts, wheat, rapeseed, sunflower seed) and a standard UK coal (Daw Mill) were tested using this method and each was found to produce unique profiles.

Abstract 55: Detection of Atrial Fibrillation Using a Smartphone Camera

Introduction: Atrial fibrillation (AF) is the most common sustained dysrhythmia in the U.S. and is associated with significant morbidity and mortality. Although new AF treatment strategies have emerged over the last decade, a major challenge facing clinicians and researchers is the paroxysmal, often short-lived, and sometimes asymptomatic nature of AF. We hypothesized that a smartphone can detect AF based on its ability to record heart beat variability by placing a fingertip on the built-in camera. Methods: In this prospective clinical investigation, we recruited 33 participants at a tertiary care medical center who presented with persistent AF prior to elective cardioversion. We recorded 2-5 minutes of pulse readings from participants’ index finger using an iPhone 4S before (AF cases), and after, electrical cardioversion (Sinus rhythm controls). With heart beat interval time series obtained from changes in color signals of a fingertip recording, we compared the results of an AF detection algorithm using Root Mean Square of Successive Difference in combination with Shannon Entropy and Sample Entropy in comparison to the gold-standard of an electrocardiogram. Results: The AF detection accuracy was 97.67% (sensitivity of 96.08% and specificity of 99.97%) for each sample when a combination of the three signal processing techniques were applied compared to the gold-standard electrocardiographic recordings. All of the 33 AF episodes were detected. In addition, we tested on 29 healthy subjects and found that the accuracy (specificity) was 99.51%. Conclusions: In this paper, we show that a smartphone can accurately detect AF based on its ability to obtain RR time series. More accessible methods to accurately detect AF are necessary, and the use of existing smartphone technologies has potentially revolutionary clinical applications for in-home monitoring for AF.

Cuffless Differential Blood Pressure Estimation Using Smart Phones

Smart phones today have become increasingly popular with the general public for their diverse functionalities such as navigation, social networking, and multimedia facilities. These phones are equipped with high-end processors, high-resolution cameras, and built-in sensors such as accelerometer, orientation-sensor, and light-sensor. According to comScore survey, 26.2% of U.S. adults use smart phones in their daily lives. Motivated by this statistic and the diverse capability of smart phones, we focus on utilizing them for biomedical applications. We present a new application of the smart phone with its built-in camera and microphone replacing the traditional stethoscope and cuff-based measurement technique, to quantify vital signs such as heart rate and blood pressure. We propose two differential blood pressure estimating techniques using the heartbeat and pulse data. The first method uses two smart phones whereas the second method replaces one of the phones with a customized external microphone. We estimate the systolic and diastolic pressure in the two techniques by computing the pulse pressure and the stroke volume from the data recorded. By comparing the estimated blood pressure values with those measured using a commercial blood pressure meter, we obtained encouraging results of 95-100% accuracy.

Atrial Fibrillation Detection Using an iPhone 4S

Atrial fibrillation (AF) affects three to five million Americans and is associated with significant morbidity and mortality. Existing methods to diagnose this paroxysmal arrhythmia are cumbersome and/or expensive. We hypothesized that an iPhone 4S can be used to detect AF based on its ability to record a pulsatile photoplethysmogram signal from a fingertip using the built-in camera lens. To investigate the capability of the iPhone 4S for AF detection, we first used two databases, the MIT-BIH AF and normal sinus rhythm (NSR) to derive discriminatory threshold values between two rhythms. Both databases include RR time series originating from 250 Hz sampled ECG recordings. We rescaled the RR time series to 30 Hz so that the RR time series resolution is 1/30 (s) which is equivalent to the resolution from an iPhone 4S. We investigated three statistical methods consisting of the root mean square of successive differences (RMSSD), the Shannon entropy (ShE) and the sample entropy (SampE), which have been proved to be useful tools for AF assessment. Using 64-beat segments from the MIT-BIH databases, we found the beat-to-beat accuracy value of 0.9405, 0.9300, and 0.9614 for RMSSD, ShE, and SampE, respectively. Using an iPhone 4S, we collected 2-min pulsatile time series from 25 prospectively recruited subjects with AF pre- and postelectrical cardioversion. Using derived threshold values of RMSSD, ShE and SampE from the MIT-BIH databases, we found the beat-to-beat accuracy of 0.9844, 0.8494, and 0.9522, respectively. It should be recognized that for clinical applications, the most relevant objective is to detect the presence of AF in the data. Using this criterion, we achieved an accuracy of 100% for both the MIT-BIH AF and iPhone 4S databases.

Specular Highlight Removal For Image-Based Dietary Assessment.

Traditional dietary assessment methods, consisting of written and orally reported methods, are not widely acceptable or feasible for everyday monitoring. The development of builtin cameras for mobile devices provides a new way of collecting dietary information by acquiring images of foods and beverages. The ability of image analysis techniques to automatically segment and identify food items from food images becomes imperative. Food images, usually consisting of plates, bowls and glasses, are often affected by lighting and specular highlights which present difficulties for image analysis. In this paper, we propose a novel single-image specular highlight removal method to detect and remove specular highlights in food images. We use independent components analysis (ICA) to separate the specular and diffuse components from the original image using only one image. This paper describes the details of the proposed model and also presents experimental results on food images to demonstrate the effectiveness of our approach.

Utility of the new McCoy laryngoscope with built-in camera and viewfinder

Utility of the new McCoy laryngoscope with built-in camera and viewfinder