Built-in Camera Research Articles

INVESTIGATION ON MULTI-SENSOR FUSION STRATEGIES FOR IMPROVED ORIENTATION DETERMINATION IN MOBILE PHONE IMAGING APPLICATIONS

Abstract. Thanks to the rapid technological progress in the field of mobile devices, smartphones are increasingly becoming valuable for science. They can serve as photogrammetric measurement devices with built-in cameras, micro-electro-mechanical systems for orientation and position assessment, as well as powerful processing units allowing field-based data acquisition and processing. This paper outlines a comprehensive investigation focusing on the accuracy and stability of smartphone camera rotation parameters determined by built-in smartphone sensors. For that purpose, the rotation parameters were measured under a range of different conditions. Four test scenarios were defined considering indoor- and outdoor measurements using three different devices being in static and dynamic modes. Furthermore, the influence of magnetic perturbations was investigated. The rotation parameters were determined from the measurements applying different sensor fusion approaches. Reference values for accuracy assessment were provided by a superior precision inertial measurement unit that measured the rotation parameters simultaneously to the smartphone in each experiment. The analysis of the smartphone-based rotation parameters, separated in the Euler angles azimuth, pitch and roll, shows average accuracies below 2° for pitch and roll. In comparison, azimuth shows significantly lower accuracies of more than 30° especially when the smartphone is in motion and when it is exposed to magnetic perturbations. In this regard, advanced multi-sensor fusion approaches were examined that handle such interferences to considerably improve the accuracy of azimuth measurements. In conclusion, a summary of accuracies and stabilities to be expected from smartphone sensors is given referring to ambient conditions and investigated sensor fusion strategies.

Sign Language Translation using Hand Gesture Detection

This Paper demonstrate a module on a sign to speech (voice) converter for auto Conversion of American sign language (ASL) to English Speech and text. It minimizes the gap of communication between speech impaired and other humans. It could be used to understand a speech impaired person’s thoughts or views which he communicates with others through its ASL gestures but failing to communicate due to a large communication gap between them. It also work as a translator for person who do not understand the sign language and allows the communication in the natural way of speaking. The proposed module is an interactive application module developed using Python and its Advanced Libraries. This module uses inbuilt camera of the system to get the images and perform analysis on those images to predict the meaning of that gesture and provide the output as text on screen and speech through speaker of the system that makes this module very much cost effective. This module recognizes one handed ASL gestures of alphabets (A-Z) with highly consistent, fairly high precision and accuracy.

Automated Plant Disease Detection using Deep Learning Architectures with Autonomous rover

Agriculture is the backbone and plays a vital role in many Asian countries. Farmers mainly depend on their agricultural produce for their living. A report says one-third of the farmers income account’s for the agricultural loss which is primarily due to plant diseases. To combat this farmers are in need of a early plant disease identification mechanism. Observation of individual plants in the farm for detecting the disease is labor-intensive and time consuming work, if the farm is vast and multiple plants are cultivated then it’s even worse. To solve such issues, current technologies like the Internet of Things (IoT) and artificial intelligence (AI) and Machine Learning (ML) are used to predict the diseases more effectively. Farmers usually detect plant diseases with the help of images captured manually and analyzed separately by experts. The proposed system renders an efficient solution for detecting multiple diseases in several plant varieties. The system is designed to detect and recognize several plant varieties, specifically pepper, grapes, and strawberry. The proposed system discovers various plant’s various diseases based on the inputs obtained by capturing images from a built-in camera present in the Autonomous rover. The rover also record’s it’s GPS location and makes a map of the entire farm traced and checked by the robot. The images are processed and are classified into their respective categories using deep learning algorithms. Convolutional neural networks the powerful methodology for image classification is the underlying principle applied. The deep learning model’s architecture namely, VGG16 and InceptionResNetV2, are used to train the model. These models are primarily made of convolutional layers. On testing, we recorded am accuracy of 93.21% was obtained from VGG16, and 95.24% from InceptionResNetV2.

Identifying Child Users via Touchscreen Interactions

With the proliferation of smart devices, children can be easily exposed to violent or adult-only content on the Internet. Without any precaution, the premature and unsupervised use of smart devices can be harmful to both children and their parents. Thus, it is critical to employ parent patrol mechanisms such that children are restricted to child-friendly content only. A successful parent patrol strategy has to be user friendly and privacy aware. The apps that require explicit actions from parents are not effective because a parent may forget to enable them, and the ones that use built-in cameras or microphones to detect child users may impose privacy violations. In this article, we propose iCare, a system that can identify child users automatically and seamlessly when users operate smartphones. In particular, iCare investigates the intrinsic differences of screen-touch patterns between child and adult users from the aspect of physiological maturity. We discover that one’s touch behaviors are related to his or her age. Thus, iCare records the touch behaviors and extracts hand geometry, finger dexterity, and hand stability features that capture the age information. We conduct experiments on 100 people including 62 children (3 to 17 years old) and 38 adults (18 to 59 years old). Results show that iCare can achieve 96.6% accuracy for child identification using only a single swipe on the screen, and the accuracy becomes 98.3% with three consecutive swipes.

3D-Printed Facet Optics: Novel Adjustable Technical Optics Inspired by Compound Eyes

Bio-inspired by compound eyes in insects, the authors identify advantages of such an optical system and propose a novel optics that combines basic principles from compound eyes with an additional technical zooming feature. The 3D-printed, bio-inspired fiber optic set-up is based on ommatidia, the small single components of compound eyes. The advantageous aspects that are transferred from the inspiring organisms are that no focusing on objects is needed and a maximum depth of focus is always achieved. Two adjustable technical features are an adjustable field of view per pixel and a zooming possibility, not found in animals. Prototypes were produced as a proof of concept. One of them was manufactured using a stereolithography 3D printer. They were positively tested with regard to the implemented features. Optional further functionalities and developments are discussed. Possible applications of the 3D-printed, bio-inspired designs are optical devices that benefit from adjusting the field of view per pixel to zooming. Suggested are novel microscopes and screens with built-in cameras enabling online eye-to-eye communication without having to concentrate on the location of a camera.

External dose-rate measurements based on smartphone CMOS sensors

In the framework of the European Project CONFIDENCE („COping with uNcertainties For Improved modelling and DEcision making in Nuclear emergenCiEs“), the characterization of two applications (“RadioactivityCounter and “GammaPix”) that enable dose-rate measurements with an ordinary smartphone was carried out. In case of a nuclear emergency, such softwares allow to use the CMOS sensor of the built-in camera as a radiation detector. This paper presents the results of the scientific investigation on the reliability of the so-called “citizen measurements” performed with 14 different mobile phones chosen among the most common brands sold in 2017. All tests were carried out at room temperature and in different reference fields at the Secondary Standard Dosimetry Laboratory of the Helmholtz Zentrum München. Investigated properties were the response at different dose-rate levels, influence of integration time, background, energy dependence and angular dependence.

Thermal Injection Measurement (TIM) Adds Accuracy to Injections and Can Assess Cold Chain Management

BackgroundInaccurate assessment of injected drug delivery may increase cost and morbidity or reduce efficacy. Yet currently most injections are evaluated solely by the formation of a visible wheal that might not truly estimate the actual area of effect. We hypothesized that thermal injection measurement (TIM) might verify appropriate temperature at the time of injection, as required for some temperature-sensitive vaccines and provide more accurate information about the area of delivery. Methods0.1 mL of either iced (n = 11) or room temperature (n = 17) methylene blue solution was injected subcutaneously in mice under anesthesia and photos taken with an iPhone 7 built-in camera and Thermal Seek Camera phone plug-in. After 5 min, true values were determined at necropsy. ResultsTIM was closer in value to the measured area at necropsy than the area of the visualized skin wheal at both ice temperature and room temperature. The difference between the true value and thermal area assessment of iced solution averaged 0.15 cm2 as compared with the difference between the true value and wheal size, which averaged 0.27 cm2 (P = 0.04). At room temperature, this was maintained for thermal and visible wheal differences, 0.23 cm2 and 0.65 cm2, respectively (P = 0.0006). ConclusionsTIM can assess temperature at the time of injection and is more accurate than visual inspection. TIM could be applied to colorless injections and areas that are hard to visualize such as scar. As a portable phone plug-in, it might be a useful adjunct to aid the evaluation of injected drug delivery including in resource-limited settings.

Cursive-Text: A Comprehensive Dataset for End-to-End Urdu Text Recognition in Natural Scene Images.

Reading text in natural scene images is an active research area in the fields of computer vision and pattern recognition as text detection, text recognition and script identification are required. In this data article, a comprehensive dataset for Urdu text detection and recognition in natural scene images is presented and analysed. To develop the dataset, more than 2500 natural scene images were captured using a digital camera and a built-in mobile phone camera. Three separate datasets for isolated Urdu character images, cropped word images and end-to-end text spotting were developed. The isolated Urdu character and cropped word images dataset contain a much larger number of samples than existing Arabic natural scene text datasets. The Urdu text spotting dataset contains images with Urdu, English and Sindhi text instances. However, the focus has been given to the Urdu text instances. The ground truths for each image in the isolated character, cropped word or text spotting datasets are provided separately. The proposed datasets can be used to perform Urdu text detection and recognition or end-to-end recognition in natural scenes. These datasets can also be helpful to develop Arabic and Persian natural scene text detection and recognition systems, as Urdu is a derived language of these scripts and has many similar letters. The datasets can also be helpful to develop multi-language translation systems, which can facilitate foreign tourists to read and translate multilingual text in natural scene images. To evaluate the datasets, state-of-the-art machine learning and deep neural networks were used to build the text detection and recognition models, where the best classification accuracies are achieved. To the best of the authors’ knowledge, this is the first dataset proposed for Urdu text detection, recognition or end-to-end text recognition in natural scene images. The aim of this data article is to present a benchmark work in the field of document analysis and recognition.Computer ScienceComputer Vision and Pattern RecognitionTablesFiguresImagesText FilesUsing a digital camera with a 20 megapixels (MP) sensor, an iPhone with a 12 MP back camera and a Samsung mobile with a 16MP back camera.RawAnalyzedEnvironmental factors such as illuminations, blurring and lighting conditions were considered while capturing images. The focus was given to the text within an image.The images in the dataset were obtained from the advertisement banners, sign-boards along the road side and streets, shop name boards, text written on the passing vehicles and walls.The images provided in this dataset were collected in different cities of Sindh, Pakistan.Summarized data are hosted with the article.The datasets and their related files are hosted in a Mendeley public data repository.DOI: https://data.mendeley.com/datasets/k5fz57zd9z/1URL: http://dx.doi.org/10.17632/k5fz57zd9z.1

MHealth spectroscopy of blood hemoglobin with spectral super-resolution.

Although blood hemoglobin (Hgb) testing is a routine procedure in a variety of clinical situations, noninvasive, continuous, and real-time blood Hgb measurements are still challenging. Optical spectroscopy can offer noninvasive blood Hgb quantification, but requires bulky optical components that intrinsically limit the development of mobile health (mHealth) technologies. Here, we report spectral super-resolution (SSR) spectroscopy that virtually transforms the built-in camera (RGB sensor) of a smartphone into a hyperspectral imager for accurate and precise blood Hgb analyses. Statistical learning of SSR enables us to reconstruct detailed spectra from three color RGB data. Peripheral tissue imaging with a mobile application is further combined to compute exact blood Hgb content without a priori personalized calibration. Measurements over a wide range of blood Hgb values show reliable performance of SSR blood Hgb quantification. Given that SSR does not require additional hardware accessories, the mobility, simplicity, and affordability of conventional smartphones support the idea that SSR blood Hgb measurements can be used as an mHealth method.

Using Diazotization Reaction to Develop Portable Liquid-Crystal-Based Sensors for Nitrite Detection.

A liquid-crystal (LC)-based sensor for detecting nitrite in aqueous solutions was developed using a diazotization reaction as the sensing mechanism. First, tetradecyl 4-aminobenzoate (14CBA) was synthesized and doped into a nematic LC, i.e., 4-cyano-4′-pentylbiphenyl (5CB). When the LC mixture was cast on a glass substrate and then immersed into an aqueous solution without nitrite, the orientation of LC was planar and the LC image was bright. In the presence of nitrite, it reacted with alkylanilines to give corresponding diazonium ions with a positive charge, which aligned at the LC/aqueous interface to cause homeotropic orientation of LC. As a result, a bright-to-dark transition of the LC image was observed. The limit of detection (LOD) of this system for nitrite is 25 μM with high selectivity. In addition, this system can work in environmental water samples such as tap water and pond water. Finally, we demonstrated that the optical signals of LC can be measured and recorded using a built-in digital camera of a smartphone, suggesting the portability of this system for on-site applications.

Short-term retention of 99mTc activity in bone scintigraphy

The objective of present study is estimation of the radioactivity held within the urinary bladder and kidneys 3 h post radiopharmaceutical (99mTc-MDP) administration, comparison being made via use of MIRDose. Using the conjugate-view method and built-in camera software Mediso XP Interview, urinary bladder and kidney radionuclide activities were calculated using patient clinical data acquired at the aforesaid 3 h post 99mTc-MDP injection juncture. The study was of a cohort of 40 patients attending for staging and cancer follow-up at the Nuclear Medicine Department, Royal Care International Hospital (RCIH), Sudan, Ethical and Research Committee approval for the study having been obtained. The radioisotope biokinetics were simulated using the Medical Internal Radiation Dose (MIRD) biokinetics model, specifically in regard to radionuclide activity in the kidneys and bladder. The results were compared with the data achieved via clinical study, showing the experimentally estimated radionuclide activity in the kidney to be greater by some 50% compared to that of simulation while the bladder radionuclide activity was found to be just 2% of the simulation result. The study also found that 0.9% of the administered radioactivity remained in the urinary bladder and kidneys 3 h post administration of the radionuclide, suggestive of low radiation dose to the urinary bladder and kidneys post 99mTc-MDP bone scans.

Monitoring Atrial Fibrillation using PPG Signals and a Smartphone

Atrial fibrillation (AF) is a common type of cardiac arrhythmia. AF is associated with increased rates of death and hospitalizations. It is also related to a degraded quality of life and reduced exercise capacity. Ageing increases the risk of developing AF, as well as hypertension and obesity. Continuous ECG monitoring is required in patients with previously diagnosed AF. Studies have also demonstrated that daily ECG monitoring increases the successful detection of silent AF among older adults. In this sense facilitating AF monitoring using portable devices such as Smartphones will increase patients life quality and could help to an early diagnosis. With this in mind in this work we present a proposal to detect AF using pulsatile photoplethysmogram (PPG) signal from a fingertip using the built-in camera lens in a smartphone. We developed an algorithm intended to sense paroxysmal AF considering resource utilization capabilities in order to be used in mobile devices with constrained.

Motivate AAC Switch Practice With Smart-Home Tech

Motivate AAC Switch Practice With Smart-Home Tech

Smartphone based iris recognition through optimized textural representation

Mobile devices have become ubiquitous nowadays and so is the need of secure access to these devices. Iris being the most reliable and hard-to-tamper biometric trait, can serve the aforementioned purpose. Iris recognition on mobile phones has become a significant and challenging task for the research community. With advancement in technology, it has now become feasible to use mobile devices’ in-built cameras to unlock the device through the user’s iris. This paper presents a convenient and efficient approach: optimal bit-transition codes (OBTC), for representing mobile iris images in a more distinctive manner. The approach is derived from the texture analysis property of 2D Gabor filters. Optimization of Gabor parameters is performed for iris images from two challenging mobile iris databases: MICHE I (which comprises of eye images acquired from three different smartphones: iPhone5, Galaxy S4 and Galaxy Tab2) and VISOB (which contains eye images acquired from iPhone5S, Samsung Note 4 and Oppo N1). After filtering, the image responses are converted to binary numbers and stored in concatenated vectors. Later, the concatenated vectors produce binary strings across the direction of concatenation and number of bit-transitions in these binary strings are encoded to form the complete feature vectors. A capacious experimentation is performed on the challenging MICHE I and VISOB iris databases. Comparison of the proposed approach with several state-of-the-art approaches clearly shows its expediency. More importantly, the proposed iris recognition approach performs at par with a commercial iris matcher, named VeriEye, which proves its usefulness.

Automatic exposure algorithms for digital photography

In this paper we deal with the problem of calculating Automatic Exposure (AE) in digital cameras. The main problem that often occurs when taking pictures is correct exposure setting. Typically, smartphones with built-in cameras, as well as “cheap” compact digital cameras do not offer possibility of manual exposure setting. The reason is that users do not have knowledge how to set the optimal exposure, or just simply do not want to do this. Therefore, it forces that user has to rely on automatic exposure algorithms implemented in the camera. Unfortunately, these algorithms often do not perform well what causes improperly exposed images. In this paper, new algorithms for automatic exposure are proposed with the special focus on minimizing overexposed areas in the images. We have implemented proposed algorithms and conducted experiments for their efficiency, comparing with some modern cameras or smartphones. Experimental verification (enhanced by statistical analysis) shows that proposed algorithms give statistically less overexposed areas than comparative AEs.

Hyperspectral imaging enabled by an unmodified smartphone for analyzing skin morphological features and monitoring hemodynamics.

We propose a novel method and system that utilizes a popular smartphone to realize hyperspectral imaging for analyzing skin morphological features and monitoring hemodynamics. The imaging system works based on a built-in RGB camera and flashlight on the smartphone. We apply Wiener estimation to transform the acquired RGB-mode images into "pseudo"-hyperspectral images with 16 wavebands, covering a visible range from 470nm to 620nm. The processing method uses weighted subtractions between wavebands to extract absorption information caused by specific chromophores within skin tissue, mainly including hemoglobin and melanin. Based on the extracted absorption information of hemoglobin, we conduct real-time monitoring experiments in the skin to measure heart rate and to observe skin activities during a vascular occlusion event. Compared with expensive hyperspectral imaging systems, the smartphone-based system delivers similar results but with very-high imaging resolution. Besides, it is easy to operate, very cost-effective and has a wider customer base. The use of an unmodified smartphone to realize hyperspectral imaging promises a possibility to bring a hyperspectral analysis of skin out from laboratory and clinical wards to daily life, which may also impact on healthcare in low resource settings and rural areas.

VISU: A 3D Printed Functional Robot for Crowd Surveillance

This article presents a novel three-dimensional printed robot for crowd surveillance called VISU. VISU captures surroundings using inbuilt camera and transmits them to the cloud. VISU's cloud architecture is connected to the criminal and suspect database provided by police authority. The captured crowd data is compared with criminal or suspect images and the authorities are alerted in case of any match. The algorithm detects the number of people captured in each frame and the authorities are prompted with the number of people in the crowd. With the help of instance segmentation, authorities can track every persons movement in the surroundings.

A Study of User Interface with Wearable Devices Based on Computer Vision

Smart wearable devices are widely used in the field of healthcare. This article presents three approaches to human–computer interaction (HCI) via computer vision and hand gestures. They are timeline-user interface, virtual keyboard user interface, and handwritten digit user interface, respectively. These user interfaces are achieved by including the image process and machine learning, such as Convolutional Neuron Networks and AdaBoost. To evaluate the approaches, a prototype of the android device with a built-in color camera was employed. The result of the handwritten digit model was significant for high accuracy of 92.9%. In our view, the results emphasize the validity of our models.

Melfalan in the treatment of proliferative vitreoretinopathy (experimental study)

To study of the effectiveness of the drug Melphalan as an antiproliferative agent during experimental proliferative vitreoretinopathy (PVR). The experimental study used data from 24 eyes of 12 Chinchilla rabbits weighing 2.5-3.0 kg, which had PVR modeled in both eyes by intravitreal injection of a culture of heterogeneous activated fibroblast cells consisting of 200,000 cells in 0.1 ml. Treatment of experimental PVR was performed 1 day after the modeling process. In the first group of animals (6 eyes), 0.02 mg of Melphalan was administered intravitreally. In the second group of animals (6 eyes), 0.005 mg of Melphalan concentrated in 0.1 ml was administered intravitreally. Left eyes in both groups remained without treatment. Animals were observed for 1 month using biomicroscopy and ophthalmoscopy. 30 days after the animals were removed from the experiment, the eyes were enucleated, fixed in 10% buffered formalin and subjected to standard histological examination. The study of paraffin sections of the eyes was performed using the microscopic system «Leica» (Leica Microsystems, Germany) with built-in digital camera at the magnification of 200-600. In groups 1 and 2 of the study in the eyes of rabbits that received treatment, PVR was absent, unlike the eyes without treatment, where PVR remained. In group 1, where the dose of Melphalan was 0.02 mg in 0.1 ml, there were changes in the RPE (retinal pigment epithelium), which was regarded as a retinotoxic effect. Glial degeneration and thinning of the retina with disappearance of the photoreceptor layer (the outer nuclear and plexiform layers) resulted from the disturbance of retinal metabolism caused by RPE destruction. In group 2, structure of the retina remained more intact: isolated foci were noted with a decrease in the volume of the outer nuclear layer, shortening of rods and cones with preservation of the inner layers of the retina. A single intravitreal injection of 0.005 mg Melphalan had a positive therapeutic antiproliferative effect on the PVR model with minimal retinotoxic changes.

Measuring Low-Order Photometric Parameters of Light Fields: Methods Exploration and Simulations

Low-order photometric parameters of light fields are important aspects determining lighting qualities but difficult to be measured. In this article, we did a systematical investigation into the performance of a cubic meter and a tetrahedron meter as well as an HDR panoramic map-based method in simultaneously recovering light density, direction, and diffuseness. Five metrics were introduced with two based on the measurement using a cubic illumination meter, two based on a tetrahedron shaped illumination meter and one based on SH decomposition of HDR panoramic maps. Furthermore, the measurement of five metrics was simulated under six HDR Panoramic maps of natural scenes in various postures which mimic the complexity of real environments. The results indicate that the measurement of low-order photometric parameters of light fields using Spherical Harmonics (i.e. latter referred to as SH) decomposition of HDR panoramic map gave the most reliable results. The tetrahedron meter based metrics gave more robust results in the measurement of light density while the cubic meter based metrics performed better when measuring light direction. This study also provides a solution for a more robust measurement of light diffuseness based on the ratio between the vector strength measured with a cubic meter and light density measured with a tetrahedron meter. With the quick development of high-speed built-in camera and mobile computing techniques, this research provides confidence in developing applications in mobile phone by capturing HDR panoramic maps or built in light meters to measure the low-order photometric parameters of light fields in 3D spaces.