Built-in Camera Research Articles

An infrared image-enhancement algorithm in simulated prosthetic vision: Enlarging working environment of future retinal prostheses.

Most existing retinal prostheses contain a built-in visible-light camera module that captures images of the surrounding environment. Thus, in case of insufficient or lack of visible light, the camera fails to work, and the retinal prostheses enter a dormant or "OFF" state. A simple and effective solution is replacing the visible-light camera with a dual-mode camera. The present research aimed to achieve two main purposes: (1) to explore whether the dual-mode camera in prosthesis recipients works under no visible-light conditions and (2) to assess its performance. To accomplish these aims, we enrolled subjects in a psychophysical experiment under simulated prosthetic vision conditions. We found that the subjects could complete some simple visual tasks, but the recognition performance under the infrared mode was significantly inferior to that under the visible-light mode. These results inspired us to develop and propose a feasible infrared image-enhancement processing algorithm. Another psychophysical experiment was performed to verify the feasibility of the algorithm. The obtained results showed that the average efficiency of the subjects completing visual tasks using our enhancement algorithm (0.014 ± 0.001) was significantly higher (p < 0.001) than that of subjects using direct pixelization (0.007 ± 0.001). We concluded that a dual-mode camera could be a feasible solution to improving the performance of retinal prostheses as the camera adapted better to the specific existing ambient light conditions. Dual-mode cameras combined with this infrared image-enhancement algorithm could provide a promising direction for the design of future retinal prostheses.

Deep-Sea Sediment and Water Simulator for Investigation of Methane Seeping and Hydrate Formation

The ubiquitous methane seeping process in the deep-sea environment could significantly influence the global methane cycle and carbon budget. Hydrate formation on the methane bubble during the seeping process is an important way for sequestrating methane during bubble migration. Uncovering the complete methane leakage process needs to reveal the methane leakage pathway and hydrate conversion mechanism. Hence, we built a deep-sea sediment and water simulator to investigate the methane seeping and hydrate formation. The simulator can mimic the deep-sea sediment and water environment with a lower sediment chamber and an upper seawater chamber. The monitoring of the bubble migration path and hydrate transformation and aggregation in the sediment chamber is realized mainly through the spatial distribution of electric resistance and temperature variations. The seawater chamber is equipped with a built-in movable camera and four external windows to observe the rising and morphological evolution of gas and hydrate bubbles. The quantitative storage and escape of CH4 gas could be realized through the measurement of multiple gas/liquid collection ports and cumulative incoming/outgoing gas volume. In addition, a movable biological liquid injection port was designed in the seawater chamber for the coupling CH4 conversion of hydrate formation and microorganism-mediated oxidation. Through the experimental test on each function of the system, the effectiveness of the device was proved. The development of this device has pioneering significance for the experimental simulation of the methane seeping process in a simulated submarine cold spring area.

Towards the Detection of GPS Spoofing Attacks against Drones by Analyzing Camera's Video Stream.

A Global Positioning System (GPS) spoofing attack can be launched against any commercial GPS sensor in order to interfere with its navigation capabilities. These sensors are installed in a variety of devices and vehicles (e.g., cars, planes, cell phones, ships, UAVs, and more). In this study, we focus on micro UAVs (drones) for several reasons: (1) they are small and inexpensive, (2) they rely on a built-in camera, (3) they use GPS sensors, and (4) it is difficult to add external components to micro UAVs. We propose an innovative method, based on the video stream captured by a drone’s camera, for the real-time detection of GPS spoofing attacks targeting drones. The proposed method collects frames from the video stream and their location (GPS coordinates); by calculating the correlation between each frame, our method can detect GPS spoofing attacks on drones. We first analyze the performance of the suggested method in a controlled environment by conducting experiments on a flight simulator that we developed. Then, we analyze its performance in the real world using a DJI drone. Our method can provide different levels of security against GPS spoofing attacks, depending on the detection interval required; for example, it can provide a high level of security to a drone flying at altitudes of 50–100 m over an urban area at an average speed of 4 km/h in conditions of low ambient light; in this scenario, the proposed method can provide a level of security that detects any GPS spoofing attack in which the spoofed location is a distance of 1–4 m (an average of 2.5 m) from the real location.

Differentiate Soybean Response to Off-Target Dicamba Damage Based on UAV Imagery and Machine Learning

The wide adoption of dicamba-tolerant (DT) soybean has led to numerous cases of off-target dicamba damage to non-DT soybean and dicot crops. This study aimed to develop a method to differentiate soybean response to dicamba using unmanned-aerial-vehicle-based imagery and machine learning models. Soybean lines were visually classified into three classes of injury, i.e., tolerant, moderate, and susceptible to off-target dicamba. A quadcopter with a built-in RGB camera was used to collect images of field plots at a height of 20 m above ground level. Seven image features were extracted for each plot, including canopy coverage, contrast, entropy, green leaf index, hue, saturation, and triangular greenness index. Classification models based on artificial neural network (ANN) and random forest (RF) algorithms were developed to differentiate the three classes of response to dicamba. Significant differences for each feature were observed among classes and no significant differences across fields were observed. The ANN and RF models were able to precisely distinguish tolerant and susceptible lines with an overall accuracy of 0.74 and 0.75, respectively. The imagery-based classification model can be implemented in a breeding program to effectively differentiate phenotypic dicamba response and identify soybean lines with tolerance to off-target dicamba damage.

Multi-ROI Spectral Approach for the Continuous Remote Cardio-Respiratory Monitoring from Mobile Device Built-In Cameras.

Heart rate (HR) and respiratory rate (fR) can be estimated by processing videos framing the upper body and face regions without any physical contact with the subject. This paper proposed a technique for continuously monitoring HR and fR via a multi-ROI approach based on the spectral analysis of RGB video frames recorded with a mobile device (i.e., a smartphone’s camera). The respiratory signal was estimated by the motion of the chest, whereas the cardiac signal was retrieved from the pulsatile activity at the level of right and left cheeks and forehead. Videos were recorded from 18 healthy volunteers in four sessions with different user-camera distances (i.e., 0.5 m and 1.0 m) and illumination conditions (i.e., natural and artificial light). For HR estimation, three approaches were investigated based on single or multi-ROI approaches. A commercially available multiparametric device was used to record reference respiratory signals and electrocardiogram (ECG). The results demonstrated that the multi-ROI approach outperforms the single-ROI approach providing temporal trends of both the vital parameters comparable to those provided by the reference, with a mean absolute error (MAE) consistently below 1 breaths·min−1 for fR in all the scenarios, and a MAE between 0.7 bpm and 6 bpm for HR estimation, whose values increase at higher distances.

Current State and Future Directions of Technologies for Music Instrument Pedagogy.

Technological advances over the past 50 years or so have resulted in the development of a succession of hardware and software systems intended to improve the quality and effectiveness of Western music instrument pedagogy during classroom instruction or individual study. These systems have aimed to provide evaluation or visualization of single or combined technical aspects by analyzing performance data collected in real time or offline. The number of such educational technologies shows an ever-increasing trend over time, aided by the wide diffusion and availability of mobile devices. However, we believe there are unrealized opportunities for modern technologies to help music students in their technical development and assist them during their practice sessions in between visits to their teachers. The ubiquity of PCs and mobile devices with built-in microphones, speakers, and cameras has inspired the development of media technologies in support of music pedagogy. They offer an attractive potential for implementing audio signal processing algorithms addressing different technical skills of the performer, providing real-time feedback, collecting data over time, and applying statistical models. Despite this potential, most available software for music instrument pedagogy remains very limited in functionality. This study provides a survey of music edTech software available, together with the methods of use, addressed technical skills, commonalities, and limitations. Results show that most current software is based on the metronome and tuner, with only a few systems that have limited abilities to follow a performance in real-time and compare it to a given score to monitor correctness of notes, intonation, and rhythm. The survey also highlights a high and under-exploited potential regarding the monitoring of other more specific technical skills, which are more instrument-dependent, but no less important, such as the control of dynamic range and clarity of the attack. This article ends with a discussion of possible directions for future development of technologies to support the practice of music students at different levels, with some consideration for the corresponding signal processing methods that can be utilized or that need advancement. By helping students to more efficiently achieve a high level of proficiency of their instruments with assistive technologies, we hope to minimize stress and afford better enjoyment of the music performance experience for all.

Development of a high-speed, phase-sensitive laser probe system for RF surface/bulk acoustic wave devices with an autofocus function for long-time continuous operation

This paper describes the implementation of an autofocus function for a laser beam in a high-speed, phase-sensitive laser probe system for RF surface/bulk acoustic wave devices. This implementation can compensate for defocus caused during continuous measurements that take dozens of hours. After a brief explanation of the system used in this work, a detailed discussion is given on the employed evaluation function indicating focus status, which is a key factor determining autofocus reliability. It is shown that the sum of the energies of Laplacians is suitable as an evaluation function, which can be calculated by an image of the probing laser spot captured by a built-in CCD camera. Then, the implementation of the autofocus function in the current system is detailed. It is confirmed that this function can adjust the focus within almost defocus conditions. Finally, it is confirmed how the implemented autofocus function works effectively to keep just-in-focus under disturbance.

Weight-Quantized SqueezeNet for Resource-Constrained Robot Vacuums for Indoor Obstacle Classification

With the rapid development of artificial intelligence (AI) theory, particularly deep learning neural networks, robot vacuums equipped with AI power can automatically clean indoor floors by using intelligent programming and vacuuming services. To date, several deep AI models have been proposed to distinguish indoor objects between cleanable litter and noncleanable hazardous obstacles. Unfortunately, these existing deep AI models focus entirely on the accuracy enhancement of object classification, and little effort has been made to minimize the memory size and implementation cost of AI models. As a result, these existing deep AI models require far more memory space than a typical robot vacuum can provide. To address this shortcoming, this paper aims to study and find an efficient deep AI model that can achieve a good balance between classification accuracy and memory usage (i.e., implementation cost). In this work, we propose a weight-quantized SqueezeNet model for robot vacuums. This model can classify indoor cleanable litters from noncleanable hazardous obstacles based on the image or video captures from robot vacuums. Furthermore, we collect videos or pictures captured by built-in cameras of robot vacuums and use them to construct a diverse dataset. The dataset contains 20,000 images with a ground-view perspective of dining rooms, kitchens and living rooms for various houses under different lighting conditions. Experimental results show that the proposed deep AI model can achieve comparable object classification accuracy of around 93% while reducing memory usage by at least 22.5 times. More importantly, the memory footprint required by our AI model is only 0.8 MB, indicating that this model can run smoothly on resource-constrained robot vacuums, where low-end processors or microcontrollers are dedicated to running AI algorithms.

Continuous blood flow visualization with laser speckle contrast imaging during neurovascular surgery.

.Significance: Laser speckle contrast imaging (LSCI) has emerged as a promising tool for intraoperative cerebral blood flow (CBF) monitoring because it produces real-time full-field blood flow maps noninvasively and label free.Aim: We aim to demonstrate the ability of LSCI to continuously visualize blood flow during neurovascular procedures.Approach: LSCI hardware was attached to the surgical microscope and did not interfere with the normal operation of the microscope. To more easily visualize CBF in real time, LSCI images were registered with the built-in microscope white light camera such that LSCI images were overlaid on the white light images and displayed to the neurosurgeon continuously in real time.Results: LSCI was performed throughout each surgery when the microscope was positioned over the patient, providing the surgeon with real-time visualization of blood flow changes before, during, and after aneurysm clipping or arteriovenous malformation (AVM) resection in humans. LSCI was also compared with indocyanine green angiography (ICGA) to assess CBF during aneurysm clipping and AVM surgery; integration of the LSCI hardware with the microscope enabled simultaneous acquisition of LSCI and ICGA.Conclusions: The results suggest that LSCI can provide continuous and real-time CBF visualization without affecting the surgeon workflow or requiring a contrast agent. The results also demonstrate that LSCI and ICGA provide different, yet complementary information about vessel perfusion.

The Asset Management and Tracking System for Technical and Vocational Education and Training (TVET) Institution Based on Ubiquitous Computing

Generally, Technical and Vocational Education and Training (TVET) has numerous types of assets used for conducting various skill-based learning activities. The failure to properly manage the TVET assets can lead to inefficient operation and administration, such as difficulty in tracking the history of the asset, location, and users. The research solved the problems by developing and implementing asset management and tracking system based on ubiquitous cloud computing for movable and fixed assets. The research activities were conducted in Politeknik Negeri Balikpapan, one of ten state-owned TVET institutions in Indonesia. This system was accessed using a web and mobile application. Quick Response (QR) code was used for asset identification to make a mobile device read the code with their built-in camera. Meanwhile, the geolocation was attached to provide the spatial information of assets’ whereabouts. Then, the research adopted the 5W1H question principle, so all aspects of asset management were collected and understood. The results show that the system helps TVET to keep track of their equipment and vital inventories in realtime. Moreover, the implementation of the system has a great impact administratively and ease in delivering instantaneous data and assets history for decisionmaking to internal and external asset auditors.

Artificial Intelligence Based Virtual Mouse

The mouse is an indispensable input device that is used in all modern-day computer systems. Input devices are high-contact surfaces that we use daily and often all day. As such the mouse is laden with germs and bacteria. . Although, wireless mice have rid us of the need for a tangled mess of cables, there still exists the need to touch the device. In view of the pandemic, this proposed system uses the in-built camera or a peripheral web camera for capturing hand movements and finger-tip detection that can perform traditional mouse functions like left-clicking, right-clicking, scrolling and cursor functions. The algorithm is machine learning based. Using deep learning, the algorithm is trained so that hands can be detected using the camera. Hence, the proposed system will prevent the spread of Covid-19 by eliminating the need for human intervention and dependency on physical devices to control the computer system.

Smartphone-Based Refractive Index Optosensing Platform Using a DVD Grating.

A low-cost, smartphone-based optical diffraction grating refractometer is demonstrated. Its principle of operation is based on the dependence of the diffraction efficiency of a DVD grating on the surrounding refractive index. The studied configuration uses the built-in LED flashlight and camera of a smartphone as a light source and a detector, respectively, to image the DVD grating diffraction pattern. No additional optical accessories, such as lenses, fibers, filters, or pinholes, are employed. The refractive index sensor exhibits a linear response in the refractive index range of 1.333–1.358 RIU (refractive index unit), with a sensitivity of 32.4 RIU−1 and a resolution of 2 × 10−3 RIU at the refractive index of water. This performance makes the proposed scheme suitable for affinity-based biosensing and a promising optosensing refractometric platform for point-of-need applications.

Visible light communication with OLEDs for D2D communications considering user movement and receiver orientations.

With the increasing use of organic light emitting diodes in lights, smart phones, wearable smartwatches, and computers, visible light-based device-to-device (D2D) communications has become more and more relevant. We propose D2D communications using smart phones' display pixels and their built-in cameras. We investigate the impact of receiver orientation and user mobility on the link performance. We derive a Gaussian model for the probability density function of the delay spread and optical path loss (OPL), and show that the channel delay spread decreases for a typical furnished room compared with an empty room, whereas the former has an increased OPL. In addition, we show that for the case of a furnished room and considering user mobility, the peak OPL values are about 64 and 62 dB, with and without considering the receiver's random orientation, respectively.

Advanced Music Player Using Sentiment Analysis

The human face is an crucial organ of an individual‘s frame and it specifically performs an crucial function in extraction of an individual’s conduct and mawkish state. Manually segregating the listing of songs and producing the best playlist primarily based totally on an individual‘s emotional capabilities is a completely tedious, time consuming, hard work in depth and upheld task. Many different algorithms were proposed and advanced for automating the playlist technology method. However the proposed current algorithms in use are computationally slow, much less correct and from time to time even require use of extra hardware like EEG or sensors. This proposed device primarily based totally on facial features extracted will generate a playlist robotically thereby lowering the time and effort concerned in rendering the method manually. The device will even consist of pointers primarily based totally on lyrics of songs in addition to questions requested from the device such as ”how become your day” aspect to person. Thus the proposed device has a tendency to lessen the computational time concerned in acquiring the outcomes and the general fee of the designed device, thereby growing the general accuracy of the device. Facial expressions are captured using a built-in camera. The accuracy of the emotion detection set of rules used with inside the device for actual time snapshots is calculated. To triumph over oversupply issues and recommending a person's particular information integrate a collaboration clear out and content material primarily based totally on software or sentiment primarily based totally on music advice dividing a workload in excessive end (server) and coffee end (mobiles) devices.

Verification of documents on the web resource using the method of text recognition

The article is devoted to the coverage of current issues related to the verification of documents on the web resource using text recognition methods. It has been established that there are various methods and software solutions for digital identity verification. It is important to introduce artificial intelligence technologies and the possibility of refinement for individual tasks, which іі increases business efficiency many times. An important drawback is the deployment of platforms on servers, due to which the financial cost of integration and renting the necessary software increases. This article discusses the verification method of checking identity documents, which plays an important role in the processes of opening new bank accounts, signing and concluding financial agreements, using artificial intelligence technologies and authentication methods A method has been developed that uses the method of integration in the form of an API. The «Tesseract» module for JavaScript was used, a module was developed for a better and faster verification method, divided into several blocks of finding coefficients to confirm the liquidity of the document. An algorithm for finding coefficients to confirm the liquidity of a document and a mathematical model of document authentication have been developed. When using the built-in camera on a mobile or portable device, as well as artificial intelligence technology and the proposed authentication algorithm, it is possible to analyze the image in order to obtain an authenticity assessment to determine whether the identification document is fake or genuine. It has been established that this method prevails due to lower financial costs in maintenance and installation, the technology does not require additional servers or software installation. Also, the proposed technology speeds up the verification process and its accuracy, working only with documents.

Impact of two energetic drinks on color stability, surface roughness and microhardness of some ceramic and hybrid materials

Objective To evaluate the impact of two energetic drinks on color stability (CS), surface roughness (SR) and microhardness (HV) of some ceramic and hybrid materials. Materials and methods A total of 72 square-shaped samples (0.8 mm in thickness, 10 mm in width and 12 mm in length) were divided into three groups (n = 24) according to the studied materials: Celtra Duo, IPS e.max CAD/CAM and Nacera Hybrid. Each group was subdivided into three subgroups (n = 8) according to the three immersion solutions applied: artificial saliva, Hype and Power horse energetic drinks. Samples were sectioned using an Isomet 4000 (Buehler, Lake Bluff, USA) and then stored in artificial saliva for 24 h before doing the baseline color, surface roughness and microhardness measurements. For determination of color change (ΔE), a reflective spectrophotometer was used. Surface roughness (SR) testing was carried out using an optical profilometer with a built-in camera and the microhardness (HV) investigation was performed with a Microhardness Digital Tester. After 60 days of immersion in energetic drinks, these measurements for color stability, surface roughness and microhardness were repeated in the same way. Data were tabulated and analyzed using a two-way ANOVA followed by one-way ANOVA and Tukey's post-hoc test with a significance level assigned at P ≤ 0.05 for all tests. Results Statistical analysis revealed significant differences between studied groups for CS (P = 0.031), SR (P = 0.012) and HV (P = 0.028). Celtra Duo recorded the lowest color change (0.57 ± 0.09) and surface roughness (0.2499 ± 0.0033) and greatest microhardness (659.62 ± 6.10) with all immersion solutions. Nacera Hybrid material showed poorer values in all investigated properties compared to Celtra Duo and IPS e.max ceramics. For all studied materials, the greatest effect of immersion solutions on CS, SR and HV was exhibited by Power horse followed by Hype and artificial saliva. Conclusions The energetic drinks studied significantly increased SR and decreased CS and VH of Nacera Hybrid. Celtra Duo and IPS e.max CAD/CAM ceramics showed greater resistance to acidic energetic drinks compared Nacera Hybrid material. Clinical significance It is essential for the dentist to reinforce the awareness of patients regarding the risks of heavy consumption of energetic drinks. Nacera Hybrid material is a promising material but should not be used in esthetic restorations.

A New Forensic Video Database for Source Smartphone Identification: Description and Analysis

In recent years, the field of digital imaging has made significant progress, so that today every smartphone has a built-in video camera that allows you to record high-quality video for free and without restrictions. On the other hand, rapidly growing internet technology has contributed significantly to the widespread use of digital video via web-based multimedia systems and mobile smartphone applications such as YouTube, Facebook, Twitter, WhatsApp, etc. However, as the recording and distribution of digital videos have become affordable nowadays, security issues have become threatening and spread worldwide. One of the security issues is identifying source cameras on videos. There are some new challenges that should be addressed in this area. One of the new challenges is individual source camera identification (ISCI), which focuses on identifying each device regardless of its model. The first step towards solving the problems is a popular video database recorded by modern smartphone devices, which can also be used for deep learning methods that are growing rapidly in the field of source camera identification. In this paper, a smartphone video database named Qatar University Forensic Video Database (QUFVD) is introduced. The QUFVD includes 6000 videos from 20 modern smartphone representing five brands, each brand has two models, and each model has two identical smartphone devices. This database is suitable for evaluating different techniques such as deep learning methods for video source smartphone identification and verification. To evaluate the QUFVD, a series of experiments to identify source cameras using a deep learning technique are conducted. The results show that improvements are essential for the ISCI scenario on video.

Image Assisted Total Stations for Structural Health Monitoring—A Review

Measuring structures and its documentation is one of the tasks of engineering geodesy. Structural health monitoring (SHM) is defined as a periodic or continuous method to provide information about the condition of the construction through the determination of measurement data and their analysis. In SHM, wide varieties of sensors are used for data acquisition. In the following, the focus is on the application of image assisted total stations (IATS). The combination of tacheometry and photogrammetric measurement offers high flexibility and precision. Different approaches of automated detecting and matching whose applications have been tested in practice are briefly explained. A distinction is made between built-in cameras (commercial) and external camera systems (prototypes). Various successful applications of IATS in the field of SHM are presented and explained.

Testing, simulation and design of offshore lined pipes under axial compression

As one of the means to inhibit offshore pipeline corrosion, lining a thin layer of stainless steel within the carbon steel pipeline provides an economical and durable design. Current design codes EN1993-1-1 and DNV–OS–F101 ignore the contribution of the liner pipes towards the cross-section capacity, which might lead to inefficient design. The focus of this study is to investigate the interaction of the two pipe layers arises during manufacture and the development of local buckling of the lined pipe under compression. Experiments to study the interaction behaviour of the liner pipe and the outer pipe, including saw tests and ring-split tests were performed. Then a series of short lined pipes were tested under axial compression, with a built-in camera to capture the development of liner wrinkling. After the tests, numerical models were developed and validated, and a series of parametric studies were conducted. The obtained test and FE results were used to assess the applicability of the codified provisions for the design of lined pipes under compression. Improved design method was developed considering the liner pipe contribution and material strain hardening. The proposed approach provides better prediction accuracies on the structural behaviours of lined pipes under axial compression.

Smartphone-based colorimetric method for determining sulfites in wine using a universal clamp sample holder and microfluidic cotton swab-based analytical device

A universal clamp sample holder was newly designed and fabricated for use with a microfluidic cotton swab-based analytical device (µCSAD) in smartphone-based colorimetric determination of sulfites in wine. In the process, sulfite (SO32−) was converted to volatile SO2 by acidification, the SO2 reduced Fe(III) to Fe(II) which finally formed the complex of Fe(II)-phenanthroline with a red color. The latter two reactions occurred on a cotton swab head hanging over the sample solution in a vial. Then, the swab was placed in the sample holder that also contained a lens and a light-emitting diode, to be photographed by the built-in camera of the smartphone. A mobile software application was designed to generate the calibration equation and evaluate the amount of sulfites in the sample. The linear ranges were 0.30–2.25 as SO2 (R2 = 0.9931) and 2.25–18.75 mg L−1 as SO2 (R2 = 0.9956). The limit of detection was 100 μg L−1. Compared to the Ripper method, the developed method showed no significant differences in measuring the sulfites in wine samples. This µCSAD is simple, cost-effectiveness, and eco-friendly. Moreover, the designed universal clamp holder and the software application can be used with multiple brands and models of smartphones.