Smartphone's Built-in Camera Research Articles

Modelling Proper and Improper Sitting Posture of Computer Users Using Machine Vision for a Human–Computer Intelligent Interactive System during COVID-19

Human posture recognition is one of the most challenging tasks due to the variation in human appearance, changes in the background and illumination, additional noise in the frame, and diverse characteristics and amount of data generated. Aside from these, generating a high configuration for recognition of human body parts, occlusion, nearly identical parts of the body, variations of colors due to clothing, and other various factors make this task one of the hardest in computer vision. Therefore, these studies require high-computing devices and machines that could handle the computational load of this task. This study used a small-scale convolutional neural network and a smartphone built-in camera to recognize proper and improper sitting posture in a work-from-home setup. Aside from the recognition of body points, this study also utilized points’ distances and angles to help in recognition. Overall, the study was able to develop two objective datasets capturing the left and right side of the participants with the supervision and guidance of licensed physical therapists. The study shows accuracies of 85.18% and 92.07%, and kappas of 0.691 and 0.838, respectively. The system was developed, implemented, and tested in a work-from-home environment.

Calculation of Heartbeat Rate and SpO2 Parameters Using a Smartphone Camera: Analysis and Testing.

Mathematical and signal-processing methods were used to obtain reliable measurements of the heartbeat pulse rate and information on oxygen concentration in the blood using short video recordings of the index finger attached to a smartphone built-in camera. Various types of smartphones were used with different operating systems (e.g., iOS, Android) and capabilities. A range of processing algorithms were applied to the red-green-blue (RGB) component signals, including mean intensity calculation, moving average smoothing, and quadratic filtering based on the Savitzky-Golay filter. Two approaches-gradient and local maximum methods-were used to determine the pulse rate, which provided similar results. A fast Fourier transform was applied to the signal to correlate the signal's frequency components with the pulse rate. We resolved the signal into its DC and AC components to calculate the ratio-of-ratios of the AC and DC components of the red and green signals, a method which is often used to estimate the oxygen concentration in blood. A series of measurements were performed on healthy human subjects, producing reliable data that compared favorably to benchmark data obtained by commercial and medically approved oximeters. Furthermore, the effect of the video recording duration on the accuracy of the results was investigated.

Digital Privacy of Assistive Technology Users with Visual Disabilities

As technology is advancing, accessibility is also taken care of seriously. Many users with visual disabilities take advantage of, for example, Microsoft's Seeing AI application (app) that is equipped with artificial intelligence. The app helps people with visual disabilities to recognize objects, people, texts, and many more via a smartphone's built-in camera. As users may use the app in recognizing personally identifiable information, user privacy should carefully be treated and considered as a top priority. Yet, little is known about the user privacy issues among users with visual disabilities, such that this study aims to address the knowledge gap by conducting a questionnaire with the Seeing AI users with visual disabilities. This study found that those with visual disabilities had a lack of knowledge about user privacy policies. It is recommended to offer an adequate educational training; thus, those with visual disabilities can be well informed of user privacy policies, ultimately leading to promoting safe online behavior to protect themselves from digital privacy and security problems.

CuCo2S4 nanozyme-based stimulus-responsive hydrogel kit for rapid point-of-care testing of uric acid.

An efficient analysis platform composed of nanozyme-based hydrogel kit and smartphone was constructed for on-site detection of uric acid (UA) in a rapid and realiable manner. CuCo2S4 nanoparticles (CuCo2S4 NPs) as a peroxidase mimic were successfully prepared and the peroxidase-like activity and catalytic mechanism were studied in detail. The hydrogen peroxide (H2O2) stimulus-responsive nanozyme-based hydrogel kit was manufactured by integrating agarose, CuCo2S4 NPs, and 3,3',5,5'-tetramethylbenzidine (TMB) into the cap of centrifuge tube. H2O2 generated via UA oxidation acts as stimulus signal, which triggers the oxidation of TMB to form blue product (oxTMB) under the catalysis of CuCo2S4 NPs, resulting in the color response of the constructed kit. The color image of the kit was captured by a smartphone built-in camera and converted into color intensity using ImageJ software, thus achieving the quantitative determination of UA. The portable kit possesses high selectivity and was used to monitor UA in human serum with satisfactory results (recovery was in the range 95.8-107.3% and RSD was not greater than 4.6%). The established sensing platform is convenient and reliable, which provides a new strategy for point-of-care testing of UA and has a broad prospect in the fields of chemical sensing and biomedical.

Smartphone-based respiratory rate estimation using photoplethysmographic imaging and discrete wavelet transform

Respiratory rate is a key vital sign that needs daily monitoring for hospital patients in general and those with respiratory conditions in particular. Moreover, it is a predictor of major heart conditions. Yet, studies have shown that it is widely neglected in hospital care due partially to the discomfort caused by the required equipment. In this paper, we propose a smartphone-based method for accurate measurement of the respiratory rate using the video of the skin surface as recorded by the smartphone built-in camera in the presence of the flash light. From this input, we use frame averaging to extract a photoplethysmographic signal of the red, green, and blue channels. Next, we apply discrete wavelet transform on the best representative photoplethysmographic signal for respiratory signal extraction and estimate the rate. Fifteen subjects participated in the testing and evaluation. The maximum absolute error was 0.67 breaths/min, whereas the root mean square error was 0.366 breaths/min. The average percentage error and average percentage accuracy using our approach were 2.2%, 97.8% respectively. A comparison with other works in the literature reveal a superior performance in terms of accuracy, ease of use, and cost.

Developing Iris Recognition System for Smartphone Security

Smartphones have become an important way to store sensitive information; therefore, users’ privacy needs to be highly protected. This can be done by using the most reliable and accurate biometric identification system available today: iris recognition. This paper develops and tests an iris recognition system for smartphones. The system uses eye images that rely on visible wavelength; these images are acquired by the smartphone built-in camera. The development of the system passes through four main phases: the first phase is the iris segmentation phase, which is done in three steps to detect the iris region from the captured image, which contains the eye and part of the face using Haar Cascade Classifier training, pupil localization, and iris localization using a Circular Hough Transform. In the second phase, the system applies normalization using a Rubber Sheet model, which converts the iris image to a fixed size pattern. In the third phase, unique features are extracted from that pattern using a Deep Sparse Filtering algorithm. Finally, in the matching phase, seven different matching techniques are investigated to decide the most appropriate one the system will use to verify the user. Two types of testing are conducted: Offline and Online tests. The BIPLab database and a collected dataset are used to measure the accuracy of the system phases and to calculate the Equal Error Rate (EER) for the whole system. The average EER is 0.18 for the BIPLab database and 0.26 for the collected dataset.

Rainfall simulation and Structure-from-Motion photogrammetry for the analysis of soil water erosion in Mediterranean vineyards

Soil water erosion is a serious problem, especially in agricultural lands. Among these, vineyards deserve attention, because they constitute for the Mediterranean areas a type of land use affected by high soil losses. A significant problem related to the study of soil water erosion in these areas consists in the lack of a standardized procedure of collecting data and reporting results, mainly due to a variability among the measurement methods applied. Given this issue and the seriousness of soil water erosion in Mediterranean vineyards, this works aims to quantify the soil losses caused by simulated rainstorms, and compare them with each other depending on two different methodologies: (i) rainfall simulation and (ii) surface elevation change-based, relying on high-resolution Digital Elevation Models (DEMs) derived from a photogrammetric technique (Structure-from-Motion or SfM). The experiments were carried out in a typical Mediterranean vineyard, located in eastern Spain, at very fine scales. SfM data were obtained from one reflex camera and a smartphone built-in camera. An index of sediment connectivity was also applied to evaluate the potential effect of connectivity within the plots. DEMs derived from the smartphone and the reflex camera were comparable with each other in terms of accuracy and capability of estimating soil loss. Furthermore, soil loss estimated with the surface elevation change-based method resulted to be of the same order of magnitude of that one obtained with rainfall simulation, as long as the sediment connectivity within the plot was considered. High-resolution topography derived from SfM revealed to be essential in the sediment connectivity analysis and, therefore, in the estimation of eroded materials, when comparing them to those derived from the rainfall simulation methodology. The fact that smartphones built-in cameras could produce as much satisfying results as those derived from reflex cameras is a high value added for using SfM.

Smartphone-Based Cardiac Rehabilitation Program: Feasibility Study.

We introduce a cardiac rehabilitation program (CRP) that utilizes only a smartphone, with no external devices. As an efficient guide for cardiac rehabilitation exercise, we developed an application to automatically indicate the exercise intensity by comparing the estimated heart rate (HR) with the target heart rate zone (THZ). The HR is estimated using video images of a fingertip taken by the smartphone’s built-in camera. The introduced CRP app includes pre-exercise, exercise with intensity guidance, and post-exercise. In the pre-exercise period, information such as THZ, exercise type, exercise stage order, and duration of each stage are set up. In the exercise with intensity guidance, the app estimates HR from the pulse obtained using the smartphone’s built-in camera and compares the estimated HR with the THZ. Based on this comparison, the app adjusts the exercise intensity to shift the patient’s HR to the THZ during exercise. In the post-exercise period, the app manages the ratio of the estimated HR to the THZ and provides a questionnaire on factors such as chest pain, shortness of breath, and leg pain during exercise, as objective and subjective evaluation indicators. As a key issue, HR estimation upon signal corruption due to motion artifacts is also considered. Through the smartphone-based CRP, we estimated the HR accuracy as mean absolute error and root mean squared error of 6.16 and 4.30bpm, respectively, with signal corruption due to motion artifacts being detected by combining the turning point ratio and kurtosis.