Front Camera Research Articles

MANGROVE HEALTH INDEX IN MARTAJASAH MANGROVE ECOTOURISM, BANGKALAN REGENCY, EAST JAVA, INDONESIA

Mangrove areas need to be considered from the condition of distribution and density of mangrove forests for future mangrove forest management planning, especially on Madura Island, especially Martajasah mangrove ecotourism. The purpose of this study is to determine the type of mangrove, canopy cover, and mangrove health index. The method used in this study is observation, which is to know the general overview of the research to be carried out. The selection of the research location was determined by the stratified random sampling method. The canopy cover on the mangrove was measured using the Hemispherical Photography method using the front camera of the OPPO Reno 6 phone with a camera resolution of 8 megapixels with a 180° angle of view at the point of taking the photo. The results of this study found 7 types of mangroves, including Sonneratia alba, Sonneratia caseolaris, Rhizophora apiculata, Rhizophora mucronata, Avicennia alba, Avicennia marina, and Lumnitzera racemosa. The percentage of canopy cover obtained an average score of 63.44% or in the solid category. This indicates that the mangrove ecosystem found in Martajasah mangrove ecotourism is in a good category. The Mangrove Health Index in Martajasah Mangrove Ecotourism is included in normal (33-66%).

Design of a Low-Cost AI System for the Modernization of Conventional Cars

Artificial intelligence techniques are beginning to be implemented in most areas. In the particular case of automobiles, new cars include integrated applications, such as cameras in different configurations, including in the rear of the car to provide assistance while reversing, as well as front and side cameras; these applications also include different configurations of sensors that provide information to the driver, such as objects approaching from different directions, such as from the front and sides. In this paper, we propose a practical and low-cost methodology to provide solutions using artificial intelligence techniques, as is the purpose of YOLO architecture, version 3, using hardware based on Nvidia’s Jetson TK1 architecture, and configurations in conventional cars. The results that we present demonstrate that these technologies can be applied in conventional cars, working with independent power to avoid causing problems in these cars, and we evaluate their application in the detection of people and cars in different situations, which allows information to be provided to the driver while performing maneuvers. The methodology that we provide can be replicated and scaled according to needs.

Design of Object Detection System for Tangkuban Parahu Volcano Monitoring Application

Indonesia has 127 active volcanoes, which have to be monitored continuously in normal, eruption, or after-eruption conditions, to minimize the effects of disaster. Therefore, we have developed a four-wheeled mobile robot for both exploration and monitoring of volcanoes. To finish its mission on uneven terrain full of obstacles, the robot should be able to detect and avoid these obstacles. Therefore, real-time object detection was designed using the YOLOv5s deep learning algorithm, which was implemented on a Raspberry Pi3 for the front camera of the robot. Before it was tested on a real volcano, the model of the algorithm was trained to be able to detect obstacles. The dataset was trained with three variations of epochs (100, 300, and 500) in sixteen batches of YOLOv5s. The last variant yielded the best results, at 63.4% mAP_0.5 and 40.4% mAP_0.5:0.95, with almost zero loss. This model was then implemented on a Raspberry Pi3 to detect trees and rocks captured by camera on Tangkuban Parahu Volcano. Most of the trees and rocks were successfully detected, with 90.9% recall, 79.9% precision, and 91.5% accuracy. Furthermore, the detection error was low, as indicated by low FP and FN numbers.

Development of intelligent traction walking plate for tension overhead construction

AbstractIn the process of tension wire construction of transmission lines, the key state quantities such as tension, inclination and spatial position of the traction walking plate directly affect the construction safety. At present, the construction personnel use visual observation to obtain the state quantity of traction walking plate in the process of tension wire construction. This monitoring method is not only inefficient, and there are personal subjective factors of judgment errors. For this reason, this paper develops a kind of intelligent traction walking plate for tension wire construction, based on the principle of resistance strain type and gyroscope to obtain the tension and inclination of the traction walking plate, respectively, using two front cameras and two rear cameras to obtain the spatial position of the traction walking plate, and finally completing the real‐time transmission of the monitoring data through the Lora technology. The developed intelligent traction walking plate was applied in Shandong Heze Yongfeng 220 kV transmission line project on 11 October 2020, which improved the construction efficiency by 15.71% while ensuring the construction safety.

SDC-Net++: End-to-End Crash Detection and Action Control for Self-Driving Car Deep-IoT-Based System.

Few prior works study self-driving cars by deep learning with IoT collaboration. SDC-Net, which is an end-to-end multitask self-driving car camera cocoon IoT-based system, is one of the research areas that tackles this direction. However, by design, SDC-Net is not able to identify the accident locations; it only classifies whether a scene is a crash scene or not. In this work, we introduce an enhanced design for the SDC-Net system by (1) replacing the classification network with a detection one, (2) adapting our benchmark dataset labels built on the CARLA simulator to include the vehicles' bounding boxes while keeping the same training, validation, and testing samples, and (3) modifying the shared information via IoT to include the accident location. We keep the same path planning and automatic emergency braking network, the digital automation platform, and the input representations to formulate the comparative study. The SDC-Net++ system is proposed to (1) output the relevant control actions, especially in case of accidents: accelerate, decelerate, maneuver, and brake, and (2) share the most critical information to the connected vehicles via IoT, especially the accident locations. A comparative study is also conducted between SDC-Net and SDC-Net++ with the same input representations: front camera only, panorama and bird's eye views, and with single-task networks, crash avoidance only, and multitask networks. The multitask network with a BEV input representation outperforms the nearest representation in precision, recall, f1-score, and accuracy by more than 15.134%, 12.046%, 13.593%, and 5%, respectively. The SDC-Net++ multitask network with BEV outperforms SDC-Net multitask with BEV in precision, recall, f1-score, accuracy, and average MSE by more than 2.201%, 2.8%, 2.505%, 2%, and 18.677%, respectively.

RescQR: Enabling Reliable Data Recovery in Screen-Camera Communication System

With an increasing number of mobile devices equipped with screens and cameras, screen-camera communication (SCC) systems enable data exchange between devices conveniently and efficiently. By encoding data with spatial and temporal diversity on a screen, multiple users with a camera can receive data without setting up a wireless network. However, as the transmitter pushes the limits of increasing throughput with a high display rate, the receiver actually suffers from a low goodput caused by composite frames. Those frames cannot be decoded correctly with existing methods. To address this problem, we propose a reliable data recovery scheme named RescQR. In RescQR, a mixture separation scheme coupled with a dedicated frame border is proposed to separate composite frames. A Viterbi-based data recovery scheme is proposed to recover data from blurred regions in composite frames. Additionally, an auto-configuration method with the help of a front camera is proposed to adjust parameters automatically according to the estimated distance between the screen and the camera. Our prototype and experiments demonstrate that RescQR achieves a data goodput of 400+kbps even with standard QR codes, which significantly outperforms previous solutions.

Self-Driving Car Using Simulator

Due to rapid technological growth in transportation, self-driving cars became the topic of concern. The main purpose of this project is to use the CNN and train the neural network in order to drive the car in autonomous mode in a simulator environment. Front camera of a car captures the images and we use those captured images in order to train the model, in short we can say we have used the concept of behavioural cloning. In behavioural cloning, the system tries to mimic the human driving behaviour by tracking the steering angle. That means a dataset is generated in the simulator by a user driven car in training mode, and the deep neural network model then drives the car in autonomous mode. In one track a car is trained and in other tracks the car drives in autonomous mode. The dataset for Track 1, which was straightforward to drive and had good road conditions, was utilized as the training set for the automobile to drive itself on Track 2, which has abrupt curves, barriers, heights, and shadows are all things to consider. Image processing and other augmentation techniques were utilized to solve this difficulty, allowing for the extraction of as many data and features as feasible. In the end, the vehicle performed admirably on Track 2. In the future, the team hopes to achieve the same level of accuracy using real-time data.

Lane detection networks based on deep neural networks and temporal information

In the past few years, the lane detection technique has become a key factor for autonomous driving systems and self-driving cars on the road. Among the various vehicle subsystems, the lane detection module is one of the essential parts of the Advanced Driver Assistance System (ADAS). Conventional lane detection approaches use machine vision algorithms to find straight lines in road scene images. However, it is challenging to identify straight or curved lane markings in complex environments. To deal with this problem, this paper presents a lane detection technique based on deep learning. It is combined with a 3D convolutional network, so the temporal information is added to the network architecture. Using the front camera images, the system can immediately detect the lane marking information ahead. Moreover, we propose an approach for improvement by adding the time axis to the network architecture. In addition to using 3D-ResNet50, the temporal convolution and spatial convolution are separated for processing. The accuracy is 91.34% improved after adding time and space split convolution with LeakyReLU. The experiments carried out using real scene images have demonstrated the feasibility of the proposed technique for applications to various complex scenes.

Minimizing optical attribute errors for a lane departure warning system using an ultra-wide-angle camera

Advanced driver assistance systems (ADAS) rely on lane departure warning (LDW) technology to enhance safety while driving. However, the current LDW method is limited to cameras with standard angles of view, such as mono cameras and black boxes. In recent times, more cameras with ultra-wide-angle lenses are being used to save money and improve accuracy. However, this has led to some challenges such as fixing optical distortion, making the camera process images faster, and ensuring its performance. To effectively implement LDW, we developed three technologies: (i) distortion correction using error functions based on the projection characteristics of optical lenses, (ii) automatic vanishing point estimation using geometric characteristics, and (iii) lane tracking and lane departure detection using constraints. The proposed technology improves system stability and convenience through automatic calculation and updating of parameters required for LDW function operation. By performing automatic distortion correction and vanishing point estimation, it has also been proven that fusion with other ADAS systems including front cameras is possible. Existing systems that use vanishing point information do not consider lens distortion and have slow and inaccurate vanishing point estimation, leading to a deterioration of system performance. The proposed method enables fast and accurate vanishing point estimation, allowing for adaptive responses to changes in the road environment.

Cognitive Impairment Detection Based on Frontal Camera Scene While Performing Handwriting Tasks

Diagnosing cognitive impairment is an ongoing field of research especially in the elderly. Assessing the health status of the elderly can be a complex process that requires both subjective and objective measures. Subjective measures, such as self-reported responses to questions, can provide valuable information about a person’s experiences, feelings, and beliefs. However, from a scientific point of view, objective measures, based on quantifiable data that can be used to assess a person’s physical and cognitive functioning, are more appropriate and rigorous. The proposed system is based on the use of non-invasive instrumentation, which includes video images acquired with a frontal camera while the user performs different handwriting tasks on a Wacom tablet. We have acquired a new multimodal database of 191 elder subjects, which has been classified by human experts into healthy and cognitive impairment users by means of the standard pentagon copying test. The automatic classification was carried out using a video segmentation algorithm through the technique of shot boundary detection, in conjunction with a Transformer neural network. We obtain a multiclass classification accuracy of 77% and two-class accuracy of 83% based on frontal camera images, which basically detects head movements during handwriting tasks. Our automatic system can replicate human classification of handwritten pentagon copying test, opening a new method for cognitive impairment detection based on head movements. We also demonstrate the possibility to identifying the handwritten task performed by the user, based on frontal camera images and a Transformer neural network.

Enhancing Reliability of Advanced Driver-Assistance Systems through Predictive Maintenance and Data-Driven Insights

The advancement of Advanced Driver Assistance Systems (ADAS) marks a pivotal evolution in automotive technology, aiming to enhance road safety and driving efficiency through a wide array of functionalities like blind spot detection, emergency braking, and adaptive cruise control. This research paper delves into the operational integrity, performance metrics, and maintenance strategies of ADAS components, underpinned by a comprehensive methodology involving data collection, pre-processing, feature engineering, machine learning model development, and rigorous validation processes. Systematic inspection of ADAS components indicates their importance in vehicle safety and reliability. The visibility, distance, speed, and steering angle of front cameras, LiDAR, radar, and ultrasonic sensors are carefully evaluated. Maintenance logs show proactive error code management, boosting efficiency. SVM, Gradient Boosting, and Random Forest machine learning models predicted ADAS component failures during validation and testing. Random Forest scored 90% accuracy, 92% precision, 88% recall, and 90% F1. Gradient Boosting was the most accurate, with 93% accuracy, 94% precision, 91% recall, and 92% F1. SVM predicted ADAS component failures with 88% accuracy, 90% precision, 85% recall, and 87% F1 score. Machine learning helps shift from reactive to proactive maintenance. Modelling sensor signal quality, actuator reaction times, error code frequencies, and maintenance intervals enables predictive maintenance and failure detection. Feature engineering builds predictive models using maintenance logs and operational KPIs. The models predict ADAS component failures, boosting vehicle safety and dependability. Using external data improves predictive maintenance models. The maintenance model's adaptability and forecast accuracy are proved by ADAS operation after traffic, accident, and manufacturer upgrades. Predictive maintenance and machine learning improve ADAS dependability and safety, the study found. Advanced analytics and data-driven insights can reduce automotive system failures, improving safety and reliability.

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

Gaze estimation aims to accurately estimate the direction or position at which a person is looking. With the development of deep learning techniques, a number of gaze estimation methods have been proposed and achieved state-of-the-art performance. However, these methods are limited to within-dataset settings, whose performance drops when tested on unseen datasets. We argue that this is caused by infinite and continuous gaze labels. To alleviate this problem, we propose using gaze frontalization as an auxiliary task to constrain gaze estimation. Based on this, we propose a novel gaze domain generalization framework named Gaze Frontalization-based Auxiliary Learning (GFAL) Framework which embeds the gaze frontalization process, i.e., guiding the feature so that the eyeball can rotate and look at the front (camera), without any target domain information during training. Experimental results show that our proposed framework is able to achieve state-of-the-art performance on gaze domain generalization task, which is competitive with or even superior to the SOTA gaze unsupervised domain adaptation methods.

Novel evaluation method for facial nerve palsy using 3D facial recognition system in iPhone

ObjectiveWhile subjective methods like the Yanagihara system and the House-Brackmann system are standard in evaluating facial paralysis, they are limited by intra- and inter-observer variability. Meanwhile, quantitative objective methods such as electroneurography and electromyography are time-consuming. Our aim was to introduce a swift, objective, and quantitative method for evaluating facial movements. MethodsWe developed an application software (app) that utilizes the facial recognition functionality of the iPhone (Apple Inc., Cupertino, USA) for facial movement evaluation. This app leverages the phone's front camera, infrared radiation, and infrared camera to provide detailed three-dimensional facial topology. It quantitatively compares left and right facial movements by region and displays the movement ratio of the affected side to the opposite side. Evaluations using the app were conducted on both normal and facial palsy subjects and were compared with conventional methods. ResultsOur app provided an intuitive user experience, completing evaluations in under a minute, and thus proving practical for regular use. Its evaluation scores correlated highly with the Yanagihara system, the House-Brackmann system, and electromyography. Furthermore, the app outperformed conventional methods in assessing detailed facial movements. ConclusionOur novel iPhone app offers a valuable tool for the comprehensive and efficient evaluation of facial palsy.

UbiHeart: A novel approach for non-invasive blood pressure monitoring through real-time facial video

Monitoring blood pressure (BP) is an essential component of evaluating cardiovascular health, aiding in the early detection and management of hypertension-related complications. Traditional methods of BP measurement often involve invasive or cumbersome devices, leading to discomfort and reduced compliance. We propose a framework to monitor BP non-invasively, analyzing the face video captured by a webcam or smartphone camera leveraging the relationship of image-based Pulse Transit Time (iPTT) and Heart Rate Variability (HRV) with BP. We have built a dataset of 90 sets of collected videos using a mobile phone front camera and BP data from a standard digital BP monitor from 12 individuals from an approved Institutional Review Board (IRB) to evaluate our system. We have got a Mean Absolute Error (MAE) of 10.35+/−2.5 mmHg for systolic BP (SBP) and 7.8+/−1.5 mmHg for diastolic BP (DBP) while using the HRV representation RMSSD. On the other hand, an MAE of 8.25+/−3.5 mmHg for SBP and 7.7+/−2.5 mmHg for DBP while using the HRV representation SDRR. Finally, we have developed a framework and built a real-time system to monitor BP as a mobile and web-based application that can facilitate early detection of trends and anomalies, allowing healthcare providers to intervene promptly and personalize treatment plans.

Calibration-Free Mobile Eye-Tracking Using Corneal Imaging.

In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system's mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that is used to estimate the user's point of gaze continuously and reliably. The system auto-calibrates the device unobtrusively. Since the user is not required to follow any special instructions to calibrate the system, they can simply put on the eye tracker and start moving around using it. Deep learning algorithms together with 3D geometric computations were used to auto-calibrate the system per user. Once the model is built, a point-to-point transformation from the eye camera to the front camera is computed automatically by matching corneal and scene images, which allows the gaze point in the scene image to be estimated. The system was evaluated by users in real-life scenarios, indoors and outdoors. The average gaze error was 1.6∘ indoors and 1.69∘ outdoors, which is considered very good compared to state-of-the-art approaches.

Surgical Training Using Augmented Reality-Based Optical Head-Mounted Displays: A Pilot Study.

While the educational benefits of the modern techniques such as virtual reality (VR) or augmented reality (AR) have been suggested, there is still a lack of reports on actual surgeons' experiences. In this study, we evaluated the effectiveness of a holographic AR-based surgical training in tonsillectomy. Two otolaryngologists, 1 trainee and 1 instructor, performed 5 tonsillectomies using an AR headset (HoloLens 2, Microsoft, USA). The trainee wore the AR headset to share the surgical view through front camera while the instructor remotely accessed the device using the Microsoft Teams program and provided real-time guidance. The AR-based surgical training offered several advantages, including direct real-time guidance for the trainee and clear instructions without disturbing the surgical process. However, there were also drawbacks, such as the front camera not always matching the trainee's view and some difficulty with focusing, depending on the depth of the oral cavity. Our study suggests that AR devices are a feasible and alternative method for surgical training. With the ability to provide clear guidance, even from a distance, this technology has the potential to revolutionize surgical training in the future.

Rail‐Lane Extraction Based on Cutting‐Edge Deep Neural Networks and Graph‐Based Segmentation

This study aims to enhance the accuracy of rail‐lane extraction, which determines the rail lane and its background at a pixel‐wise level from an ego‐perspective image captured by the front camera of a train. For this purpose, cutting‐edge deep neural network (DNN)‐based semantic segmentation models, MANet and UNet++, were applied. In addition, a method that effectively enlarges the rail lane extracted by DNN‐based segmentation was proposed based on thoroughly analyzing the issues of DNN‐based segmentation. Our experiment, which employed 601 ego‐perspective images, demonstrated that the proposed method using the cutting‐edge DNN‐based segmentation method increased the balanced accuracy by 6.66% compared with the conventionally employed ERFNet in the literature. Further discussions and useful analyses are presented herein. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Product Development of Smartphone Accessories for Solo Creators for Self-Monitoring during Video Recording for Social Media Content

This paper aims to explore the development of new products from the Industrial Design Approach as smartphone accessories that help solo content creators as a new career in the new media revolution. Recognizing the limitations of recording via smartphones today, with features such as slow-motion video and better quality missing from existing smartphone selfie (front) cameras, additional accessory product projects were developed as a solution for solo creators of video content on social media to record themselves they use the back of the smartphone camera. This product is developed from problem-solving and supported with features that can give users a better smartphone screen display while recording videos.

Estimation of Lateral Velocity and Cornering Stiffness in Vehicle Dynamics Based on Multi-Source Information Fusion

<div>To address the challenge of directly measuring essential dynamic parameters of vehicles, this article introduces a multi-source information fusion estimation method. Using the intelligent front camera (IFC) sensor to analyze lane line polynomial information and a kinematic model, the vehicle’s lateral velocity and sideslip angle can be determined without extra sensor expenses. After evaluating the strengths and weaknesses of the two aforementioned lateral velocity estimation techniques, a fusion estimation approach for lateral velocity is proposed. This approach extracts the vehicle’s lateral dynamic characteristics to calculate the fusion allocation coefficient. Subsequently, the outcomes from the two lateral velocity estimation techniques are merged, ensuring rapid convergence under steady-state conditions and precise tracking in dynamic scenarios. In addition, we introduce a tire parameter online adaptive module (TPOAM) to continually update essential tire parameters such as cornering stiffnesses, with its effectiveness demonstrated through DLC and slalom simulation tests. Using a dual extended Kalman filter (DEKF) observer, the article allows for joint estimation of vehicle states and tire parameters. Ultimately, we offer a cost-effective estimation method of vital dynamic vehicle parameters to support the motion control module in autonomous driving.</div>

Accuracy and Usability of Smartphone-Based Distance Estimation Approaches for Visual Assistive Technology Development.

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1-3 meters, including machine learning (CoreML), infrared grid distortion (IR_self), light detection and ranging (LiDAR_back), and augmented reality room-tracking on the front (ARKit_self) and back-facing cameras (ARKit_back). Results: For accuracy in the image center, all approaches had <±2.5 cm average error, except CoreML which had ±5.2-6.2 cm average error at 2-3 meters. In the periphery, all approaches were more inaccurate, with CoreML and IR_self having the highest average errors at ±41 cm and ±32 cm respectively. For usability, CoreML fared favorably with the lowest central processing unit usage, second lowest battery usage, highest field-of-view, and no specialized sensor requirements. Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV.