Commercial Smartphone Research Articles

Accurate 3D Shape, Displacement and Deformation Measurement Using a Smartphone.

The stereo-digital image correlation technique using two synchronized industrial-grade cameras has been extensively used for full-field 3D shape, displacement and deformation measurements. However, its use in resource-limited institutions and field settings is inhibited by the need for relatively expensive, bulky and complicated experimental set-ups. To mitigate this problem, we established a cost-effective and ultra-portable smartphone-based stereo-digital image correlation system, which only uses a smartphone and an optical attachment. This optical attachment is composed of four planar mirrors and a 3D-printed mirror support, and can split the incoming scene into two sub-images, simulating a stereovision system using two virtual smartphones. Although such a mirror-based system has already been used for stereo-image correlation, this is the first time it has been combined with a commercial smartphone. This publication explores the potential and limitations of such a configuration. We first verified the effectiveness and accuracy of this system in 3D shape and displacement measurement through shape measurement and in-plane and out-of-plane translation tests. Severe thermal-induced virtual strains (up to 15,000 με) were found in the measured results due to the smartphone heating. The mechanism for the generation of the temperature-dependent errors in this system was clearly and reasonably explained. After a simple preheating process, the smartphone-based system was demonstrated to be accurate in measuring the strain on the surface of a loaded composite specimen, with comparable accuracy to a strain gauge. Measurements of 3D deformation are illustrated by tracking the deformation on the surface of a deflating ball. This cost-effective and ultra-portable smartphone-based system not only greatly decreases the hardware investment in the system construction, but also increases convenience and efficiency of 3D deformation measurements, thus demonstrating a large potential in resource-limited and field settings.

Linux page fault analysis in android systems

In modern smartphones, system performances are tightly related to a variety of underlying subsystems. In particular, internal storage, along the years, has become crucial because it is extensively used to access content relevant to the system and, finally, to the end user. To understand its role in a commercial Android smartphone and to evaluate its effects on the User experience, within the context of a real usage, we analyzed Linux page fault handling, a critical mechanism that puts pressure on storage devices and may cause system inefficiencies. A kernel tracing technique has been conceived for real-time measurement of Android applications and services on commercial smartphones. The experimental results presented in this work are derived from the use of this kernel tracing on a 64-bit Android smartphone, equipped with a ufs storage subsystem. The main subject of the study is major page fault handling, a kernel mechanism behind many end-user actions recognized at industry level as source of possible performance deterioration in a smartphone. The analysis shows that major page fault handling is dominated by read accesses to ufs (between 30% and 40% of the total time), and that the related storage traffic is significantly affected by the ReadAhead mechanism, which is not always efficient.

Battery-Less NFC Sensor for pH Monitoring

In this paper, we present a low-cost battery-less color sensor based on Near Field Communication (NFC). This system consists of a color sensor connected to a low-power microcontroller connected to an NFC chip. The energy needed to power up the chipset is retrieved from the magnetic field generated by the reader, i.e., a commercial smartphone. The low cost of NFC chips affords the opportunity to manufacture low-cost tags with sensing capability. We have developed a mobile application to simultaneously present the data and send them to a database in the cloud. The system has been tested with several pH reactive strips. The pH was determined via color measurement using the HSV color space. A simple linear relationship was found between the Hue parameter and the pH. Comparison with the pH determination via smartphone camera showed that the accuracy was greater when the NFC colorimeter was used.

A Pose Awareness Solution for Estimating Pedestrian Walking Speed

Pedestrian walking speeds (PWS) can be used as a “body speedometer” to reveal health status information of pedestrians and positioning indoors with other locating methods. This paper proposes a pose awareness solution for estimating pedestrian walking speeds using the sensors built in smartphones. The smartphone usage pose is identified by using a machine learning approach based on data from multiple sensors. The data are then coupled tightly with an adaptive step detection solution to estimate the pedestrian walking speed. Field tests were carried out to verify the advantages of the proposed algorithms compared to existing solutions. The test results demonstrated that the features extracted from the data of the smartphone built-in sensors clearly reveal the characteristics of the pose pattern, with overall accuracy of 98.85% and a kappa statistic of 98.46%. The proposed walking speed estimation solution, running in real-time on a commercial smartphone, performed well, with a mean absolute error of 0.061 m/s, under a challenging walking process combining various usage poses including texting, calling, swinging, and in-pocket modes.

Experimental Indoor Visible Light Positioning Systems With Centimeter Accuracy Based on a Commercial Smartphone Camera

We consider indoor positioning based on visible light where the receiver adopts a commercial smartphone camera. Two positioning approaches are proposed, either the light emitting diode (LED) positions are known or not. When the LED positions are known, the LED light signal intensities are measured to identify the identity of each LED in the image. Triangular similarity is adopted to estimate the receiver position. When the LED positions are unknown, we develop a shift rotation model on the receiver movement and further propose a novel indoor positioning algorithm. The algorithm estimates the rotation center in the image instead of treating the image center as the rotation center, leading to reduced positioning error. To achieve high accuracy, a location reference point is set such that the positioning errors of multiple LEDs have little effect. According to the experimental results, the average positioning error can be reduced to 1 cm, which outperforms the reported experimental results with receiver-ceiling distance larger than 2 m.

A simple location-tracking app for psychological research

Location data gathered from a variety of sources are particularly valuable when it comes to understanding individuals and groups. However, much of this work has relied on participants’ active engagement in regularly reporting their location. More recently, smartphones have been used to assist with this process, but although commercial smartphone applications are available, these are often expensive and are not designed with researchers in mind. To overcome these and other related issues, we have developed a freely available Android application that logs location accurately, stores the data securely, and ensures that participants can provide consent or withdraw from a study at any time. Further recommendations and R code are provided in order to assist with subsequent data analysis.

Assessing a novel way to measure step count while walking using a custom mobile phone application.

IntroductionWalking speed has been associated with many clinical outcomes (e.g., frailty, mortality, joint replacement need, etc.). Accurately measuring walking speed (stride length x step count/time) typically requires significant clinician/staff time or a gait lab with specialized equipment (i.e., electronic timers or motion capture). In the present study, our goal was to measure “step count” via smartphones through novel software and to compare with step tracking software that come standard with iOS and Android smartphones as a first step in walking speed measurement.MethodsA separate calibration and validation data collection was performed. Individuals in the calibration collection (n = 5) walked 20m at normal and slow speed (<1.0 m/s). Appropriate settings for the novel mobile application were chosen to measure step count. Individuals in the validation (n = 52) collection walked at 6m, 10m, and 20m at normal and slow walking speeds. We compared step difference (absolute difference) from observed step counts to native step tracking software and our novel software derived step counts. We used generalized estimated equation adjusted (participant level) negative binomial regression models of absolute step difference from observed step counts, to determine optimal settings (calibration) and subsequently to gauge performance of the shake algorithm settings and native step tracking software across different distances and speeds (validation).ResultsFor iOS/iPhone 6, when compared to observed step count, the shake service (software driven approach) significantly outperformed the embedded native step tracking software across all distances at slow speed, and for short distance (6m) at normal speed. On the Android phone, the shake service outperformed the native step tracking software at slow speed at 6 meters and 20 meters, while its performance eclipsed the native step tracking software only at 20 meters at normal speed.DiscussionOur software based approach outperformed native step tracking software across various speeds and distances and carries the advantage of having adjustable measurement parameters that can be further optimized for specific medical conditions. Such software applications will provide an effective way to capture standardized data across multiple commercial smartphone devices, facilitating the future capture of walking speed and other clinically important performance parameters that will influence clinical and home care in the era of value based care.

MagIO: Magnetic Field Strength Based Indoor- Outdoor Detection with a Commercial Smartphone.

A wide range of localization techniques has been proposed recently that leverage smartphone sensors. Context awareness serves as the backbone of these localization techniques, which helps them to shift the localization technologies to improve efficiency and energy utilization. Indoor-outdoor (IO) context sensing plays a vital role for such systems, which serve both indoor and outdoor localization. IO systems work with collaborative technologies including the Global Positioning System (GPS), cellular tower signals, Wi-Fi, Bluetooth and a variety of smartphone sensors. GPS- and Wi-Fi-based systems are power hungry, and their accuracy is severed by limiting factors like multipath, shadowing, etc. On the other hand, various built-in smartphone sensors can be deployed for environmental sensing. Although these sensors can play a crucial role, yet they are very less studied. This research aims at investigating the use of ambient magnetic field data alone from a smartphone for IO detection. The research first investigates the feasibility of utilizing magnetic field data alone for IO detection and then extracts different features suitable for IO detection to be used in machine learning-based classifiers to discriminate between indoor and outdoor environments. The experiments are performed at three different places including a subway station, a shopping mall and Yeungnam University (YU), Korea. The training data are collected from one spot of the campus, and testing is performed with data from various locations of the above-mentioned places. The experiment involves Samsung Galaxy S8, LG G6 and Samsung Galaxy Round smartphones. The results show that the magnetic data from smartphone magnetic sensor embody enough information and can discriminate the indoor environment from the outdoor environment. Naive Bayes (NB) outperforms with a classification accuracy of 83.26%, as against Support vector machines (SVM), random induction (RI), gradient boosting machines (GBM), random forest (RF), k-nearest neighbor (kNN) and decision trees (DT), whose accuracies are 67.21%, 73.38%, 73.40%, 78.59%, 69.53% and 68.60%, respectively. kNN, SVM and DT do not perform well when noisy data are used for classification. Additionally, other dynamic scenarios affect the attitude of magnetic data and degrade the performance of SVM, RI and GBM. NB and RF prove to be more noise tolerant and environment adaptable and perform very well in dynamic scenarios. Keeping in view the performance of these classifiers, an ensemble-based stacking scheme is presented, which utilizes DT and RI as the base learners and naive Bayes as the ensemble classifier. This approach is able to achieve an accuracy of 85.30% using the magnetic data of the smartphone magnetic sensor. Moreover, with an increase in training data, the accuracy of the stacking scheme can be elevated by 0.83%. The performance of the proposed approach is compared with GPS-, Wi-Fi- and light sensor-based IO detection.

Clustering and Symbolic Regression for Power Consumption Estimation on Smartphone Hardware Subsystems

The subsystem in a smartphone means its hardware components, such as the CPU, GPU, and screen. Accurately estimating subsystem power consumption of commercial smartphones is necessary for applicable to wide research areas. Current subsystem power estimation techniques are mostly based on power models, resulting in considerable errors for various types of power consumption behaviors. These include (1) asynchrony between the measured power consumption and the corresponding workload statistics, and (2) nonlinearity concerning CPU idle states, pixels colors of AMOLED screen, and GPU workload statistics. In this study, we propose a novel utilization-based, subsystem power estimation method for a smartphone, namely Clustering and Symbolic Regression (CSR) that takes these power consumption behaviors into account so as to increase power estimation accuracy. To address asynchrony, we cluster the subsystem workload statistics into synchronous and asynchronous groups by employing affinity propagation clustering. To address nonlinearity, we employ symbolic regression for fitting measured power consumptions with respect to subsystem workload statistics. We compare our approach with various power estimation methods, Linear Regression Model (LM), Genetic Programming (GP), and Support Vector Regression (SVR). The results show Mean Absolute Percentage Error (MAPE) reduction between 23.61 and 42.55 percent on the estimated power consumption of a simple (Nexus S) and complex (Galaxy S4) smartphone subsystems.

Autonomous power management in mobile devices using dynamic frequency scaling and reinforcement learning for energy minimization

Embedded systems execute applications that execute hardware differently depending on the computation task, generating time-varying workloads. Energy minimization can be reached by using the low-power central processing unit (CPU) frequency for each workload. We propose an autonomous and online approach, capable of reducing energy consumption from adaptation to workload variations even in an unknown environment. In this approach, we improved the AEWMA algorithm into a new algorithm called AEWMA-MSE, adding new functionality to detect workload changes and demonstrating why it is better to use statistical analysis for real user cases in a mobile environment. Also, a new power model for mobile devices based on k-NN algorithm for regression was proposed and validated proving to have a better trade-off between execution time and precision than neural networks and linear regression-based models. AEWMA-MSE and the proposed power model are integrated into a novel algorithm for energy management based on reinforcement learning that suitably selects the appropriate CPU frequency based on workload predictions to minimize energy consumption. The proposed approach is validated through simulation by using real smartphone data from an ARM Cortex A7 processor used in a commercial smartphone. Our proposal proved to have an improvement in the Q-learning cost function and can effectively minimize the average energy consumption by 21% and up to 29% when compared to the already existing approaches.

Smartphone near infrared monitoring of plant stress

The most widely used method for monitoring plant stress is the use of near infrared (NIR) spectrophotometry to calculate normalized difference vegetation index (NDVI), as defined by [NIR reflectance − red reflectance]/[NIR reflectance + red reflectance]. NDVI measures the chlorophyll absorption in the red spectrum relative to the scattering by cellular structure in NIR, and has been used to monitor vegetation health and subsequently its stress from aerial or satellite images. Rather than using an NIR spectrophotometer or an NIR camera that is rather expensive, we attempted to use a commercial smartphone, utilizing its (potentially unintended) ability in recognizing near infrared (NIR) color. Some of the most recent versions of smartphones have eliminated the NIR block filters on their cameras, and able to recognize NIR in their red pixels of CMOS array. Through attaching an inexpensive high pass filter at 800 nm to a smartphone camera, we were able to collect the NIR reflectance (with a high pass optical filter) and the red reflectance (without a filter), enabling NDVI assessments. This method was verified by measuring the NDVI values from a series of chlorophyll solutions, and showed a strong linear correlation with R2 = 0.948, corroborating the smartphone’s ability in evaluating NDVI. Using the leaves from three different plant species, the NDVI values were evaluated using the smartphone and compared with the plants’ chlorophyll contents using acetone extraction and subsequent spectrophotometry. A good linear relationship was found with R2 = 0.88–0.92. We further evaluated the NDVI values against the plants’ water contents (measured by oven-drying), showing the non-linear relationship with the NDVI saturation above 50% water content. The assay time was almost instantaneous, requiring only a smartphone and a high pass filter, thus allowing inexpensive, easy-to-use, rapid, and early prediction of plant stress that can be used for field and household applications.

Estimation of pedestrian crowds' properties using commercial tablets and smartphones

ABSTRACTIn this study, an alternative method to estimate velocity and density in pedestrian crowds is presented. The proposed approach uses angular velocity and acceleration measurements obtained from inertial sensors to judge amount of body motion and therefore estimate walking velocity. Taking into consideration practical aspects, commercial devices (generic tablets and smartphones) are used to measure inertial quantities showing application potentials but also limitations in regard to accuracy. Several experiments have been performed in increasingly realistic scenarios to assess the suitability of the method for public pedestrian spaces. Results show that estimation for speed and density is very accurate when position and orientation of the devices are known. Estimations are still reliable in more heterogeneous situations employing different devices, with accuracy increasing with the number of pedestrians being surveyed. The proposed approach may help getting a general picture of pedestrian crowds' condition over large areas with greater employment flexibility.

Facial recognition, emotion and race in animated social media

Facial recognition systems are increasingly common components of commercial smart phones such as the iPhone X and the Samsung Galaxy S9. These technologies are also increasingly being put to use in consumer-facing social media video-sharing applications, such as Apple’s animoji and memoji, Facebook Messenger’s masks and filters and Samsung’s AR Emoji. These animations serve as technical phenomena translating moments of affective and emotional expression into mediated socially legible forms. Through an analysis of these objects and the broader literature on digital animation, this paper critiques the ways these facial recognition systems classify and categorize racial identities in human faces. The paper considers the potential both for racializing logics as part of these systems of classification, and how data regarding emotional expression gathered through these systems might interact with identity-based forms of classification.

Plasmonic Interferometer Array Biochip as a New Mobile Medical Device for Cancer Detection.

We report a plasmonic interferometer array (PIA) sensor and demonstrate its ability to detect circulating exosomal proteins in real-time with high sensitivity and low cost to enable the early detection of cancer. Specifically, a surface plasmon wave launched by the nano-groove rings interferes with the free-space light at the output of central nano-aperture and results in an intensity interference pattern. Under the single-wavelength illumination, when the target exosomal proteins are captured by antibodies bound on the surface, the biomediated change in the refractive index between the central aperture and groove rings causes the intensity change in transmitted light. By recording the intensity changes in real-time, one can effectively screen biomolecular binding events and analyze the binding kinetics. By integrating signals from multiple sensor pairs to enhance the signal-to-noise ratio, superior sensing resolutions of 1.63×10-6 refractive index unit (RIU) in refractive index change and 3.86×108 exosomes/mL in exosome detection were realized, respectively. Importantly, this PIA sensor can be imaged by a miniaturized microscope system coupled with a smart phone to realize a portable and highly sensitive healthcare device. The sensing resolution of 9.72×109 exosomes/mL in exosome detection was realized using the portable sensing system building upon a commercial smartphone.

Calibration and Noise Identification of a Rolling Shutter Camera and a Low-Cost Inertial Measurement Unit.

A low-cost inertial measurement unit (IMU) and a rolling shutter camera form a conventional device configuration for localization of a mobile platform due to their complementary properties and low costs. This paper proposes a new calibration method that jointly estimates calibration and noise parameters of the low-cost IMU and the rolling shutter camera for effective sensor fusion in which accurate sensor calibration is very critical. Based on the graybox system identification, the proposed method estimates unknown noise density so that we can minimize calibration error and its covariance by using the unscented Kalman filter. Then, we refine the estimated calibration parameters with the estimated noise density in batch manner. Experimental results on synthetic and real data demonstrate the accuracy and stability of the proposed method and show that the proposed method provides consistent results even with unknown noise density of the IMU. Furthermore, a real experiment using a commercial smartphone validates the performance of the proposed calibration method in off-the-shelf devices.

Induction machine fault detection using smartphone recorded audible noise

This study presents induction machine fault detection possibilities using smartphone recorded audible noise. Acoustic and audible noise analysis for fault detection is a well-established technique; however, specialised equipment for diagnostic purposes is often very expensive and difficult to operate. To overcome this obstacle, a simple pre-diagnostic procedure, using hand-held smartphones is proposed. Different faults of the three-phase squirrel cage induction machine such as various numbers of broken rotor bars and dynamic rotor eccentricity are inflicted to the machine and the resulting audible signals are recorded in laboratory circumstances using two widely available commercial smartphones. The analysis is performed on audible noise and compared with the results of mechanical vibrations measurements, recorded by vibration sensors. Rotational speed frequency and twice-line frequency are used as diagnostic indicators of faults. A simple neural network is composed and probabilities of fault detection using such diagnostic measures are presented. The necessity for further study as well as further implementation and method refinement necessity is pointed out.

Privacy Issues in Smartphone Applications: An Analysis of Headache/Migraine Applications.

Headache diaries are a mainstay of migraine management. While many commercial smartphone applications (apps) have been developed for people with migraine, little is known about how well these apps protect patient information and whether they are secure to use. We sought to assess whether there are privacy issues surrounding apps so that physicians and patients could better understand what medical information patients are providing to the app companies, and the potential privacy implications of how the app companies (and other third parties) might use that information. We conducted a systematic search of the most popular "headache" and "migraine" apps and developed a database of the types of data the apps requested for input by the user and whether the apps had clear privacy policies. We also examined the content of the privacy policies. Twenty-nine apps were examined (14 diary apps, 15 relaxation apps). Of the diary applications, 79% (11/14) had visible privacy policies. Of the diary apps with privacy policies, all (11/11) stated whether or not the app collects and stores information remotely. A total of 55% (6/11) stated that some user data were used to serve targeted advertisements. A total of 11/15 (73%) of the relaxation apps had privacy policies. Headache apps shared information with third parties, posing privacy risks partly because there are few legal protections against the sale or disclosure of data from medical apps to third parties.

Dynamic Frequency Scaling Regarding Memory for Energy Efficiency of Embedded Systems

Memory significantly affects the power consumption of embedded systems as well as performance. CPU frequency scaling for power management could fail in optimizing the energy efficiency without considering the memory access. In this paper, we analyze the power consumption and energy efficiency of an embedded system that supports dynamic scaling of frequency for both CPU and memory access. The power consumption of the CPU and the memory is modeled to show that the memory access rate affects the energy efficiency and the CPU frequency selection. Based on the power model, a method for frequency selection is presented to optimize the power efficiency which is measured using Energy-Delay Product (EDP). The proposed method is implemented and tested on a commercial smartphone to achieve about 3.3% - 7.6% enhancement comparing with the power management policy provided by the manufacturer in terms of EDP.

Treatment Plans for Various Neurologic Conditions Include the Use of Smartphone Applications: An observational study of the privacy issues related to commercial smartphone applications using headache applications as an example (P4.145)

Treatment Plans for Various Neurologic Conditions Include the Use of Smartphone Applications: An observational study of the privacy issues related to commercial smartphone applications using headache applications as an example (P4.145)

Lateral Position Error Reduction Using Misalignment-Sensing Coils in Inductive Power Transfer Systems

Inductive power transfer (IPT) remains one of the most common ways to achieve wireless power transmission, operating on the same electromagnetic principle as electrical transformers but with an air core instead of an iron core. IPT has been implemented in many applications, including wireless charging of consumer products like smartphones and electric vehicles. However, one challenge with using IPT is ensuring precise alignment between the primary and secondary coils of the system so that maximum power transfer can take place. In this paper, the use of misalignment-sensing coils to detect and correct lateral misalignments in an IPT system is modeled and tested. The sensing coils exploit magnetic-field symmetry to give a nonlinear measurement of misalignment direction and magnitude. Experiments using such sensing coils give a misalignment sensing resolution better than 1 mm when applied to a commercial smartphone wireless charger. Voltage readings from the sensing coils are used for closed-loop control of an experimental two-dimensional coil positioner that can reduce lateral misalignments as large as 35 mm to less than 1.4 mm, allowing for effective power transfer. A similar sensing coil concept can be used to reduce lateral misalignments in scaled IPT systems, such as electric-vehicle wireless chargers.