Built-in Camera Research Articles

AI Virtual Mouse Using Hand Gesture Recognition

The mouse is one of HCI's (Human Computer Interaction) incredible inventions. A wireless mouse or a Bluetooth mouse still uses devices because a battery is needed for power and a dongle is needed to connect the mouse to the PC.Therefore, they are not totally device-free. The proposed AI virtual mouse system can solve this problem by using a webcam or built-in camera to record hand motions and identify hand tips using computer vision. The system's algorithm makes use of the machine learning algorithm. The computer can be controlled digitally and cando left-click, right-click, scrolling, and computer cursor functions based on hand motions without the need for a physical mouse. Deep learning is the basis for the hand detection method. As a result, by eliminating human contact and the requirement for external devices to operate the computer, the suggested strategy will stop the spread of COVID-19.

The Radical Scope of Tesla's Data Hoard: Every Tesla is providing reams of sensitive data about its driver's life

You Wouldn't Have Seen a Single Tesla in the Beidaihe District of Qinhuangdao,China, this summer. Officials banned Elon Musk's popular electric vehicles from this beachfront resort for two months while it hosted the Communist Party's annual retreat, presumably fearing what the cars' built-in cameras might capture and send back to the United States. • This might seem to be undue paranoia, but Tesla vehicles do collect an enormous amount of information. That's well illustrated by events in Florida, where Tesla faced a negligence lawsuit after two young men died in the fiery crash of a 2014 Model S. The father of the teen driver sued Tesla for producing a battery prone to ignite. As part of its defense, the company submitted a historical speed analysis showing that in the months before the crash the car had been routinely driven dangerously fast. This information was quietly captured by the car and uploaded to Tesla's servers.

IoT based guided fire fighting vehicle

Fire accidents are occurring frequently nowadays in domestic and industrial environments with unpredictable climatic changes and human negligence. The proposed prototype is designed to detect such fire accidents and to extinguish the fire by remote operation of the user, and thereby reducing the risk of fire fighters involved. The movement of the vehicle is dictated by the combination of sensor-based inputs and user guidance. When the fire is detected by the fire detection sensor, the signal is immediately passed to the controller. GSM modules will be activated alerting the user about the fire accident. Once the alert is received, the user can guide the vehicle to the required location with the help of Blynk software & in-built camera module to extinguish the fire. The servo motor connected with the firefighting hose can control the direction and volume of water to be sprayed. The controls of the servo motors are with the user. The proposed firefighting vehicle is designed to function using Internet of Things (IoT), thereby enabling the user to control the fire from anywhere.

An artificial intelligence-enabled smartphone app for real-time pressure injury assessment.

The management of chronic wounds in the elderly such as pressure injury (also known as bedsore or pressure ulcer) is increasingly important in an ageing population. Accurate classification of the stage of pressure injury is important for wound care planning. Nonetheless, the expertise required for staging is often not available in a residential care home setting. Artificial-intelligence (AI)-based computer vision techniques have opened up opportunities to harness the inbuilt camera in modern smartphones to support pressure injury staging by nursing home carers. In this paper, we summarise the recent development of smartphone or tablet-based applications for wound assessment. Furthermore, we present a new smartphone application (app) to perform real-time detection and staging classification of pressure injury wounds using a deep learning-based object detection system, YOLOv4. Based on our validation set of 144 photos, our app obtained an overall prediction accuracy of 63.2%. The per-class prediction specificity is generally high (85.1%–100%), but have variable sensitivity: 73.3% (stage 1 vs. others), 37% (stage 2 vs. others), 76.7 (stage 3 vs. others), 70% (stage 4 vs. others), and 55.6% (unstageable vs. others). Using another independent test set, 8 out of 10 images were predicted correctly by the YOLOv4 model. When deployed in a real-life setting with two different ambient brightness levels with three different Android phone models, the prediction accuracy of the 10 test images ranges from 80 to 90%, which highlight the importance of evaluation of mobile health (mHealth) application in a simulated real-life setting. This study details the development and evaluation process and demonstrates the feasibility of applying such a real-time staging app in wound care management.

Performance of QR Code Detectors near Nyquist Limits.

For the interacting with real world, augmented reality devices need lightweight yet reliable methods for recognition and identification of physical objects. In that regard, promising possibilities are offered by supporting computer vision with 2D barcode tags. These tags, as high contrast and visually well-defined objects, can be used for finding fiducial points in the space or to identify physical items. Currently, QR code readers have certain demands towards the size and visibility of the codes. However, the increase of resolution of built-in cameras makes it possible to identify smaller QR codes in the scene. On the other hand, growing resolutions cause the increase to the computational effort of tag location. Therefore, resolution reduction in decoders is a common trade-off between processing time and recognition capabilities. In this article, we propose the simulation method of QR codes scanning near limits that stem from Shannon’s theorem. We analyze the efficiency of three publicly available decoders versus different size-to-sampling ratios (scales) and MTF characteristics of the image capture subsystem. The MTF we used is based on the characteristics of real devices, and it was modeled using Gaussian low-pass filtering. We tested two tasks—decoding and locating-and-decoding. The findings of the work are several-fold. Among others, we identified that, for practical decoding, the QR-code module should be no smaller than 3–3.5 pixels, regardless of MTF characteristics. We confirmed the superiority of Zbar in practical tasks and the worst recognition capabilities of OpenCV. On the other hand, we identified that, for borderline cases, or even below Nyquist limit where the other decoders fail, OpenCV is still capable of decoding some information.

Digital Privacy of Assistive Technology Users with Visual Disabilities

As technology is advancing, accessibility is also taken care of seriously. Many users with visual disabilities take advantage of, for example, Microsoft's Seeing AI application (app) that is equipped with artificial intelligence. The app helps people with visual disabilities to recognize objects, people, texts, and many more via a smartphone's built-in camera. As users may use the app in recognizing personally identifiable information, user privacy should carefully be treated and considered as a top priority. Yet, little is known about the user privacy issues among users with visual disabilities, such that this study aims to address the knowledge gap by conducting a questionnaire with the Seeing AI users with visual disabilities. This study found that those with visual disabilities had a lack of knowledge about user privacy policies. It is recommended to offer an adequate educational training; thus, those with visual disabilities can be well informed of user privacy policies, ultimately leading to promoting safe online behavior to protect themselves from digital privacy and security problems.

Naive Bayes Algorithm Mining Mobile Phone Trojan Crime Clues

After the mobile phone virus infects the mobile phone, it can transmit the real-time information of the user to the designated place set by the virus through the built-in recorder and camera on the mobile phone, thereby causing information leakage. With the rapid development of the Internet, the penetration rate of mobile terminals is also increasing day by day. As an emerging mobile terminal, smart phones have now fully occupied the market. With this trend, the importance of mobile phone information security is also increasing day by day. How to prevent mobile phone virus has gradually become an important issue. Trojan horse crime cases have different manifestations and behavioral characteristics from traditional cases. They have the characteristics of low crime cost, high income, high concealment, novel criminal methods, and great difficulty in detection, which brings greater difficulties to the public security organs in their investigation and detection. And the current research on mobile phone virus behavior is still in the preliminary stage, and some existing detection models can only target random networks. Trojan horses, viruses, and malicious software for smartphones have sprung up like mushrooms after rain, seriously infringing on the data security of mobile communication terminals, such as mobile phones and causing incalculable losses to users. This paper proposes a naive Bayesian algorithm to mine the clues of the criminal cases of mobile phone Trojans. It helps detect and discover new viruses at the beginning of an attack, allowing them to be more effectively defended and contained. And based on the feature set data extracted from the network data packets, it conducts an in-depth analysis of the current business behaviors of mobile phone Trojans, such as propagation and implantation, remote control, leakage of user privacy information, and malicious ordering, and extracts its behavior characteristics. Thus, unknown mobile Trojan horses that are taking place can be detected. The experimental results of the naive Bayesian classification algorithm proposed in this paper show that the algorithm improves the accuracy of mobile phone Trojan virus mining by 28%, which plays a significant role.

Assessment of Pancreatic and Hepatic Histological Changes in Streptozotocin Induced Diabetic Rats Fed with Morizella Juice

Background: Hibiscus sabdariffa and Moringa oleifera has been reported to have ant diabetic effect on hyperglycemic animals. However, there are no reports on its histological and physiological changes in diabetic animals.
 Objective: To assess the pancreatic and hepatic histological changes in streptozotocin-induced diabetic rats fed with Morizella juice.
 Methods: A Completely Randomized Block Design (CRBD) was used for this study, 40 adult male Wistar rats, 9 to 10 weeks with a bodyweight 190g – 250g were selected for the study. The animals were divided into four main groups (A, B, C, and D), each group was further subdivided into two groups with 5 rats. The experiments were divided into acute and sub-acute tests of 10 animals per group. At the end of the experiment, animals were sacrificed, pancreatic and liver tissues were harvested and histologically processed. Sections were examined and photographed using a light microscope (Leica® DM 750) with an in-built camera ((Icc50 HD-47142065) and the number of Islet cells was measured in 40 high-power fields.
 Results: Microscopic structures of the pancreas revealed that the number of Islets of Langerhans was significantly higher in the Morizella treated groups at the dosages of 50mg/kg and 100 mg/kg body weight. The microscopic structure of the liver in both treated groups presented with normal histological architecture when compared with both control groups
 Conclusion: There was a dose-dependent increase of islets of Langerhans cells in the acute and sub-acute group treated with Morizella extracts compared to the control group. Additionally, the liver architecture was reversed back to the normal in Morizella-treated group.

A Convolutional Neural-Network-Based Training Model to Estimate Actual Distance of Persons in Continuous Images.

Distance and depth detection plays a crucial role in intelligent robotics. It enables drones to understand their working environment to avoid collisions and accidents immediately and is very important in various AI applications. Image-based distance detection usually relies on the correctness of geometric information. However, the geometric features will be lost when the object is rotated or the camera lens image is distorted. This study proposes a training model based on a convolutional neural network, which uses a single-lens camera to estimate humans’ distance in continuous images. We can partially restore depth information loss using built-in camera parameters that do not require additional correction. The normalized skeleton feature unit vector has the same characteristics as time series data and can be classified very well using a 1D convolutional neural network. According to our results, the accuracy for the occluded leg image is over 90% at 2 to 3 m, 80% to 90% at 4 m, and 70% at 5 to 6 m.

Virtual Mouse And Keyboard For Computer Interaction

Mouse and keyboard are the wonderful inventions of Human-Computer Interaction (HCI) technology. Currently, wireless mouse or a Bluetooth mouse still uses devices and is not free of devices completely since it uses a battery for power and a dongle to connect it to the PC. In the proposed virtual mouse and keyboard system, this limitation can be overcome by employing webcam or a built-in camera for capturing of hand gestures and coloured object. Method used for Hand gesture recognition is based on Convolutional Neural Networks (CNNs).The computer can be controlled virtually and can perform left click, right click, scrolling functions,volumecontrol,brightness control and other computer cursor function without the use of a physical mouse.A virtual keyboard is controlled by tracking a coloured object and it is based on image processing technique. The proposed system will mostly can use on the conference to present topics using projector and avoid COVID-19 spread by eliminating the human intervention and dependency of devices to control the computer.Main goal is to make the interaction between human and computer as natural as the interactionbetween humans.

Distributed Visual Crowdsensing Framework for Area Coverage in Resource Constrained Environments.

Visual crowdsensing applications using built-in cameras in smartphones have recently attracted researchers’ interest. Making the most out of the limited resources to acquire the most helpful images from the public is a challenge in disaster recovery applications. Proposed solutions should adequately address several constraints, including limited bandwidth, limited energy resources, and interrupted communication links with the command center or server. Furthermore, data redundancy is considered one of the main challenges in visual crowdsensing. In distributed visual crowdsensing systems, photo sharing duplicates and expands the amount of data stored on each sensor node. As a result, if any node can communicate with the server, then more photos of the target region would be available to the server. Methods for recognizing and removing redundant data provide a range of benefits, including decreased transmission costs and energy consumption overall. To handle the interrupted communication with the server and the restricted resources of the sensor nodes, this paper proposes a distributed visual crowdsensing system for full-view area coverage. The target area is divided into virtual sub-regions, each of which is represented by a set of boundary points of interest. Then, based on the criteria for full-view area coverage, a specific data structure theme is developed to represent each photo with a set of features. The geometric context parameters of each photo are utilized to extract the features of each photo based on the full-view area coverage criteria. Finally, data redundancy removal algorithms are implemented based on the proposed clustering scheme to eliminate duplicate photos. As a result, each sensor node may filter redundant photographs in dispersed contexts without requiring high computational complexity, resources, or global awareness of all photos from all sensor nodes inside the target area. Compared to the most recent state-of-the-art, the improvement ratio of the added values of the photos provided by the proposed method is more than 38%. In terms of traffic transfer, the proposed method requires fewer data to be transferred between sensor nodes and between sensor nodes and the command center. The overall reduction in traffic exceeds 20% and the overall savings in energy consumption is more than 25%. It was evident that in the proposed system, sending photos between sensor nodes, as well as between sensor nodes and the command center, consumes less energy than existing approaches due to the considerable amount of photo exchange required. Thus, the proposed technique effectively transfers only the most valuable photos needed.

CuCo2S4 nanozyme-based stimulus-responsive hydrogel kit for rapid point-of-care testing of uric acid.

An efficient analysis platform composed of nanozyme-based hydrogel kit and smartphone was constructed for on-site detection of uric acid (UA) in a rapid and realiable manner. CuCo2S4 nanoparticles (CuCo2S4 NPs) as a peroxidase mimic were successfully prepared and the peroxidase-like activity and catalytic mechanism were studied in detail. The hydrogen peroxide (H2O2) stimulus-responsive nanozyme-based hydrogel kit was manufactured by integrating agarose, CuCo2S4 NPs, and 3,3',5,5'-tetramethylbenzidine (TMB) into the cap of centrifuge tube. H2O2 generated via UA oxidation acts as stimulus signal, which triggers the oxidation of TMB to form blue product (oxTMB) under the catalysis of CuCo2S4 NPs, resulting in the color response of the constructed kit. The color image of the kit was captured by a smartphone built-in camera and converted into color intensity using ImageJ software, thus achieving the quantitative determination of UA. The portable kit possesses high selectivity and was used to monitor UA in human serum with satisfactory results (recovery was in the range 95.8-107.3% and RSD was not greater than 4.6%). The established sensing platform is convenient and reliable, which provides a new strategy for point-of-care testing of UA and has a broad prospect in the fields of chemical sensing and biomedical.

Optical Camera Communication in Vehicular Applications: A Review

Recently, substantial development is observed in the area of Internet of Vehicles owing to the application of wireless communication technologies. Majority of these technologies are based on radio frequency (RF); however, RF spectra are overly congested and regulated, and hence, insufficient to support massive data traffic in the future. In recent times, optical camera communication (OCC) that uses a light-emitting diode (LED) as a transmitter and a camera as a receiver has been deemed an excellent solution for future intelligent transportation systems. As a communication medium, OCC mostly uses visible light, the spectrum of which is vast, completely free, and unregulated. The current outdoor environment is heavily crammed with LED infrastructures, and most vehicles have built-in cameras, rendering OCC immensely promising. OCC is highly secured, supports mobility, and can achieve an excellent bit-error rate. However, the data rate obtained using OCC is not as high as that obtained using other RF-based systems; therefore, its reliability in fast-changing channels is still under research. This review article discusses the applications of the OCC system in vehicle-to-vehicle and vehicle-to-infrastructure (or vice versa) networks; to the best of our knowledge, this is the first extensive review dedicated to the above topic. Herein, we provide a general overview of OCC standardization in IEEE and ISO in recent years. Then, we explain the general principles of OCC, including channel characteristics, region of interest signaling, and modulation schemes particularly considered in vehicular communications. Additionally, we present a comprehensive overview of the effects of mobility, noise, and interference in OCC. Finally, the challenges and future opportunities in OCC are outlined.

The Effect of Lighting and Photograph Exposure on Perceived Attractiveness.

To assess the effect of various lighting conditions and photograph exposures on perceived attractiveness. In the first experiment, 5 variably exposed photographs were taken of 10 subjects using a consistent lighting condition (45° superior box light). In the second experiment, 10 subjects were photographed under variable lighting conditions with consistent exposure: 1) 90° overhead box light, 2) ring light, 3) 45° superior box light, 4) built-in camera flash, 5) 2 straight on box lights, each 45° from midline, and 6) natural light. Participants were instructed to maintain a neutral expression, were placed in front of a standardized blue-gray background, and were photographed during a single session. Photographs were imported into an online survey platform (Qualtrics 2020) and displayed in random order. Volunteer survey respondents were instructed to rate the subject's attractiveness on a scale of 0 to 10. Between the two experiments, a total of 22,000 scored photographs were included in the analysis. Mixed ANOVA and pairwise comparisons with Bonferroni correction were used to compare between- and within-subject ratings. Lighting condition had a significant impact on perceived attractiveness ( p < 0.001), with the 90° overhead box light achieving lower scores and the 45° superior box light yielding greater scores of attractiveness relative to the other conditions. Photograph exposure did not have a significant impact on subjective attractiveness ( p = 1.000). Our findings suggest that perceived attractiveness is enhanced when a 45° superior box light is used for illumination, and attractiveness is reduced when 90° overhead exposure is utilized. Exposure did not play a prominent role in perceived attractiveness.

What Apps Do Postsecondary Students with Attention Deficit Hyperactivity Disorder Actually Find Helpful for Doing Schoolwork? An Empirical Study

Attention Deficit Hyperactivity Disorder (ADHD) experts and individuals with ADHD have made many recommendations concerning mobile apps that could potentially help college students succeed. But do students know about these recommended apps? Do they find them useful? How do students use mobile apps for completing schoolwork? To answer these questions we carried out two empirical studies. In study 1, 35 Canadian postsecondary students who self-reported ADHD and 74 students without disabilities completed an online LimeSurvey questionnaire and indicated which of 20 expert recommended schoolwork-related apps they had tried and which they liked. In Study 2, nine students with ADHD specified how they used their technologies to complete schoolwork. Results indicate that students with and without ADHD were familiar with only 13 of the 20 apps recommended by experts, and that they liked only 11. For completing academic work, the most popular apps were built-in smartphone camera and recording apps. Students also found Microsoft, Google, Pomodoro and Kahoot apps helpful. Discord was the most popular app for collaboration with classmates. Results show that students with and without ADHD found the same apps and technologies helpful. To the best of our knowledge this is the first study to actually query students with ADHD about their app use and preferences. We conclude that disability service providers, academic administrators and access technologists need to stay up-to-date about general use mobile apps to enable them to make appropriate recommendations to help students with ADHD succeed in college.

ISVD-Based Advanced Simultaneous Localization and Mapping (SLAM) Algorithm for Mobile Robots

In the case of simultaneous localization and mapping, route planning and navigation are based on data captured by multiple sensors, including built-in cameras. Nowadays, mobile devices frequently have more than one camera with overlapping fields of view, leading to solutions where depth information can also be gathered along with ordinary RGB color data. Using these RGB-D sensors, two- and three-dimensional point clouds can be recorded from the mobile devices, which provide additional information for localization and mapping. The method of matching point clouds during the movement of the device is essential: reducing noise while having an acceptable processing time is crucial for a real-life application. In this paper, we present a novel ISVD-based method for displacement estimation, using key points detected by SURF and ORB feature detectors. The ISVD algorithm is a fitting procedure based on SVD resolution, which removes outliers from the point clouds to be fitted in several steps. The developed method removes these outlying points in several steps, in each iteration examining the relative error of the point pairs and then progressively reducing the maximum error for the next matching step. An advantage over relevant methods is that this method always gives the same result, as no random steps are included.

Drowsiness Detection Application

Every year, hundreds of people die in car accidents around the world, with the primary cause being driver inattention. A sleepiness detection system will aid in the reduction of this accident and the saving of countless lives all around the world. To defend this problem, we propose a methodology based on Machine learning (ML) that illustrates drowsiness detection as a task to detect an object. It will detect and localize whether the eyes are open or close based on the real time video stream of drivers. Through the live video streaming, a frame is extracted for image processing. Images are captured typically at a fix frame rate of 20fps based on brightness and camera quality. The OpenCV and Dlib support the android platform to detect the faces from the live frame and predict the driver drowsiness. The suggested method aims to improve precision and computational efficiency. It's also cost-effective because it can handle video feeds in real-time and doesn't require any expensive hardware. It is also affordable as it can process incoming video streams in real-time and does not need any expensive hardware support. There only needs a inbuilt camera of android and provide a alert sound when the system predict the driver is drowsy. After that, it send SMS to the police station with these predicted information.

485-P: DFUCare Deep-Learning–Based Mobile App for Diabetic Foot Ulcer Monitoring

In 2021, 1 out of adults suffered from diabetes mellitus globally. A major complication resulting from this is diabetic foot ulcers (DFU) , which can lead to amputation if improperly managed. Current clinical approaches to DFU have significant limitations, such as the high cost involved in the diagnosis and time-consuming care. In light of this, we have developed DFUCare, a mobile app that uses the built-in camera and deep learning to diagnose and quantify patients' diabetic foot ulcers. DFUCare is trained on an extensive foot dataset containing over 5000 images of normal and infected DFU patients. For binary classification of infected and non-infected patients, we use a combination of transfer and ensemble learning. Using transfer learning, DFUCare uses the pretrained InceptionResNetV2 model for deep learning derived feature extraction. With an ensemble approach, we compare the deep learning derived feature extraction results to handcrafted color and textural feature extraction results to come up with a final prediction of infected vs. non-infected patients. In the end, we achieved top-level results with a binary accuracy of .9014, AUC of .9437, specificity of .9023, and precision of .8901. In addition, using standardized metrics and computer vision algorithms, DFUCare can determine the dimensions of the DFU to quantify the severity over time. With all of the information stored in the cloud, doctors and patients can use the share feature in the mobile app to send medical reports to anyone in the world. In the future, we are planning to add clinical, biological, and epidemiological features along with macroscopic image features to improve the performance of the tool in predicting infection and healing. This novel approach has the potential to deliver a paradigm shift in diabetic foot care among patients by being a cost-effective, remote, and convenient healthcare solution. Disclosure V.Sendilraj: None. W.C.Pilcher: None. M.K.Bhasin: None.

Smart Surveillance

Abstract: Every individual in today's society has a need for a safe and reliable system. The most commonly used closed-circuit television (CCTV) or video surveillance systems are being implemented everywhere: hospitals, warehouses, parking lots, and buildings. However, this highly effective system has a cost disadvantage. Therefore, a cost-effective system is required. This project proposes to use a security camera with night vision using OpenCV. This is a cost-effective method. Images are captured and processed frame by frame. When a person is detected, the image is saved and an email is sent. The accuracy of this system is about 83%. It also uses the improved properties of the built-in camera. So the image is captured by the camera and sent to processing for face and person detection using OpenCV. Then, whether the detected person is known (visitor) or (stranger) () the detected face is compared against the database, and based on the output an email is generated and sent to the user. Security cameras can be controlled by custom AI. assistant. AI features include turning security cameras on and off with voice control. Accordingly, a low-cost security system can be provided. Keywords: OpenCV, Computer Vision, Tkinter, CCTV, Surveillance, Noise, Motion Artificial Intelligence

Virtual Mouse using Hand Gesture to avoid Infectious Diseases

One of the most astonishing Human-Computer Interaction (HCI) inventions is the mouse. A wireless mouse or a Bluetooth mouse now require devices and are not completely free of gadgets because they rely on a battery for power and a dongle to connect to a PC. This problem can be solved in the proposed AI virtual mouse system by capturing hand motions and recognising hand tips with computer vision utilising a webcam or a built-in camera. In the system's algorithm, the machine learning algorithm is used. Hand movements can be used to control the computer remotely, and it can perform left click, right click, scrolling operations, and computer cursor functions without the need for a hardware mouse. Deep learning is used by the system to detect hands. As a result, by eliminating human influence, the proposed technique will prevent the spread of COVID-19 and other infectious diseases. In this paper, we have discussed the implementation of AI virtual keyboard and mouse system that uses computer vision to perform mouse functions in a computer by using hand gestures and fingertip detection. Keywords —Machine Learning, OpenCV, Media pipe, Hand tracking, Python