Low-cost Camera Research Articles

DOES: A Deep Learning-Based Approach to Estimate Roll and Pitch at Sea

The use of Attitude and Heading Reference Systems (AHRS) for orientation estimation is now common practice in a wide range of applications, e.g., robotics and human motion tracking, aerial vehicles and aerospace, gaming and virtual reality, indoor pedestrian navigation and maritime navigation. The integration of the high-rate measurements can provide very accurate estimates, but these can suffer from errors accumulation due to the sensors drift over longer time scales. To overcome this issue, inertial sensors are typically combined with additional sensors and techniques. As an example, camera-based solutions have drawn a large attention by the community, thanks to their low-costs and easy hardware setup; moreover, impressive results have been demonstrated in the context of Deep Learning. This work presents the preliminary results obtained by DOES, a supportive Deep Learning method specifically designed for maritime navigation, which aims at improving the roll and pitch estimations obtained by common AHRS. DOES recovers these estimations through the analysis of the frames acquired by a low-cost camera pointing the horizon at sea. The training has been performed on the novel ROPIS dataset, presented in the context of this work, acquired using the FrameWO application developed for the scope. Promising results encourage to test other network backbones and to further expand the dataset, improving the accuracy of the results and the range of applications of the method as a valid support to visual-based odometry techniques.

Automatic Object Detection, Labelling, and Localization by Camera’s Drone System

This work explores the designing a system of an automated unmanned aerial vehicles (UAV( for objects detection, labelling, and localization using deep learning. This system takes pictures with a low-cost camera and uses a GPS unit to specify the positions. The data is sent to the base station via Wi-Fi connection.
 The proposed system consists of four main parts. First, the drone, which was assembled and installed, while a Raspberry Pi4 was added and the flight path was controlled. Second, various programs that were installed and downloaded to define the parts of the drone and its preparation for flight. In addition, this part included programs for both Raspberry Pi4 and servo, along with protocols for communication, video transmission, and sending and receiving signals between the drone and the computer. Third, a real-time, modified, one dimensional convolutional neural network (1D-CNN) algorithm, which was applied to detect and determine the type of the discovered objects (labelling). Fourth, GPS devices, which were used to determine the location of the drone starting and ending points . Trigonometric functions were then used for adjusting the camera angle and the drone altitude to calculate the direction of the detected object automatically.
 According to the performance evaluation conducted, the implemented system is capable of meeting the targeted requirements.

Low-Cost Camera-Based Smart Surveillance System for Detecting, Recognizing, and Tracking Masked Human Face

A surveillance system is still the most exciting and practical security system to prevent crime effectively. The primary purpose of this system is to recognize the identity of the face caught by the camera. With the advancement of the Internet of things, surveillance systems were implemented on edge devices such as the low-cost Raspberry mobile camera. It raises the challenge of unstructured image/video where the video contains low quality, blur, and variations of human poses. The challenge is increasing because people used to wear a mask during the Covid -19 pandemic. Therefore, we proposed developing an all-in-one surveillance system with face detection, recognition, and face tracking capabilities. This system integrated three modules: MTCNN face detector, VGGFace2 face recognition, and Discriminative Single-Shot Segmentation (D3S) tracker to create a system capable of tracking the faces of people caught on surveillance camera. We also train new face mask data to recognize and track. This system obtains data from the Raspberry Pi camera and processes images on the cloud as a mobile sensor approach. The proposed system successfully implemented and obtained competitive results in detection, recognition, and tracking under an unconstrained surveillance camera.

Effect of low-cost transcranial magnetic stimulation navigation on hotspot targeting and motor evoked potential variability in the biceps brachii.

Transcranial magnetic stimulation (TMS) can monitor or modulate brain excitability. However, reliability of TMS outcomes depends on consistent coil placement during stimulation. Neuronavigated TMS systems can address this issue, but their cost limits their use outside of specialist research environments. The objective was to evaluate the performance of a low-cost navigated TMS approach in improving coil placement consistency and its effect on motor evoked potentials (MEPs) when targeting the biceps brachii at rest and during voluntary contractions. We implemented a navigated TMS system using a low-cost 3D camera system and open-source software environment programmed using the Unity 3D engine. MEPs were collected from the biceps brachii at rest and during voluntary contractions across two sessions in ten non-disabled individuals. Motor hotspots were recorded and targeted via two conditions: navigated and conventional. The low-cost navigated TMS system reduced coil orientation error (pitch: 1.18°±1.2°, yaw: 1.99°±1.9°, roll: 1.18°±2.2° with navigation, versus pitch: 3.7°±5.7°, yaw: 3.11°±3.1°, roll: 3.8°±9.1° with conventional). The improvement in coil orientation had no effect on MEP amplitudes and variability. The low-cost system is a suitable alternative to expensive systems in tracking the motor hotspot between sessions and quantifying the error in coil placement when delivering TMS. Biceps MEP variability reflects physiological variability across a range of voluntary efforts, that can be captured equally well with navigated or conventional approaches of coil locating.

Recognition of Manual Welding Positions from Depth Hole Image Remotely Sensed by RGB-D Camera

The proportion of welding work in total man-hours required for shipbuilding processes has been perceived to be significant, and welding man-hours are greatly affected by working posture. Continuous research has been conducted to identify the posture in welding by utilizing the relationship between man-hours and working posture. However, the results that reflect the effect of the welding posture on man-hours are not available. Although studies on posture recognition based on depth image analysis are being positively reviewed, welding operation has difficulties in image interpretation because an external obstacle caused by arcs exists. Therefore, any obstacle element must be removed in advance. This study proposes a method to acquire work postures using a low-cost RGB-D camera and recognize the welding position through image analysis. It removes obstacles that appear as depth holes in the depth image and restores the removed part to the desired state. The welder’s body joints are extracted, and a convolution neural network is used to determine the corresponding welding position. The restored image showed significantly improved recognition accuracy. The proposed method acquires, analyzes, and automates the recognition of welding positions in real-time. It can be applied to all areas where image interpretation is difficult due to obstacles.

Applications of Pose Estimation in Human Health and Performance across the Lifespan.

The emergence of pose estimation algorithms represents a potential paradigm shift in the study and assessment of human movement. Human pose estimation algorithms leverage advances in computer vision to track human movement automatically from simple videos recorded using common household devices with relatively low-cost cameras (e.g., smartphones, tablets, laptop computers). In our view, these technologies offer clear and exciting potential to make measurement of human movement substantially more accessible; for example, a clinician could perform a quantitative motor assessment directly in a patient’s home, a researcher without access to expensive motion capture equipment could analyze movement kinematics using a smartphone video, and a coach could evaluate player performance with video recordings directly from the field. In this review, we combine expertise and perspectives from physical therapy, speech-language pathology, movement science, and engineering to provide insight into applications of pose estimation in human health and performance. We focus specifically on applications in areas of human development, performance optimization, injury prevention, and motor assessment of persons with neurologic damage or disease. We review relevant literature, share interdisciplinary viewpoints on future applications of these technologies to improve human health and performance, and discuss perceived limitations.

Utilization of programmable cameras for web-based sensing and control of daylight in buildings

The concept of integrating programmable low-cost cameras into the office infrastructure and BMS for real-time, web-based sensing and control of the luminous environment in buildings is presented in this study. Experiments were conducted to evaluate the potential of predicting the luminance field perceived by an office occupant using a programmable calibrated HDR camera installed at the rear side of a computer monitor or on the wall behind the occupant, for a variety of sky conditions and shading options. The generated luminance maps using Python scripts with OpenCV packages were further processed to extract daylighting and glare metrics using Evalgare. The results showed that: (i) among the different camera resolutions that were compared, the 330x330 resolution was selected as the best option to balance between accurate capturing of visual environment and comfort and computational efficiency; (ii) a camera sensor embedded on the rear side of a computer screen could capture interior visual conditions consistently similarly to those viewed by the occupant, except for sunny conditions without proper shading protection. This prototype study paves the way for luminance monitoring and daylight control using programmable low-cost camera sensors embedded into the office infrastructure.

FusionVLP: The Fusion of Photodiode and Camera for Visible Light Positioning

The previous Photodiode-only (PD-only) VLP systems might not provide an accurate positioning solution in an interference environment. Meanwhile, the previous camera-only VLP systems might need a specific camera with a high resolution and a large Field Of View (FOV). This article proposes a Visible Light Positioning (VLP) system (FusionVLP) based on the fusion of the measurements from a standard PD and a low-cost camera to compensate their individual deficiencies, thus improving VLP accuracy and achieving great potential for low-cost deployment. FusionVLP has an Ensemble Kalman Filter (EnKF) module for real-time positioning and a Fixed-Lag Ensemble Kalman Smoother (FLEnKS) module for semi-real-time positioning for different application requirements. Furthermore, a new two-step adjusted PD Received Signal Strength (RSS) model with adaptive parameter training is proposed to mitigate the model error of the traditional model, thus improving VLP accuracy. To further enhance the positioning accuracy, a novel positioning method based on a typical rectangular ceiling pattern is proposed, thus providing a camera-based specific position measurement for data fusion and improving the positioning by 36.27% (2D) and 35.61% (3D). The field experiments in an interference environment show that FusionVLP achieves the accuracy of 8.67 cm (2D)/12.15 cm (3D) using FLEnKS.

Non-Intrusive Luminance Mapping via High Dynamic Range Imaging and 3-D Reconstruction

Continuous luminance monitoring is challenging because high-dynamic-range cameras are expensive, they need programming, and are intrusive when placed near the occupants’ field-of-view. A new semi-automated and non-intrusive framework is presented for monitoring occupant-perceived luminance using a low-cost camera sensor and Structure-from- Motion (SfM)-Multiview Stereo (MVS) photogrammetry pipeline. Using a short video and a few photos from the occupant position, the 3D space geometry is automatically reconstructed. Retrieved 3D context enables the back-projection of the camera-captured luminance distribution into 3D spaces that are in turn re-projected to occupant-FOVs. The framework was tested and validated in a testbed office. The re-projected luminance field showed with good agreement with luminance measured at the occupant position. The new method can be used for non-intrusive luminance monitoring integrated with daylighting control applications.

High-Speed and Continuous-Wave Programmable Luminescent Tags Based on Exclusive Room Temperature Phosphorescence (RTP).

Most materials recently developed for room temperature phosphorescence (RTP) lack in practical relevance due to their inconvenient crystalline morphology. Using amorphous material systems instead, programmable luminescent tags (PLTs) based on organic biluminescent emitter molecules with easy processing and smooth sample shapes are presented recently. Here, the effective quenching of the emitter's RTP by molecular oxygen (O2) and the consumption of the excited singlet O2 through a chemical reaction represent the central features. With customized activation schemes, high‐resolution content can be written and later erased multiple times into such films, providing a versatile yet simple photonic platform for information storage. However, two important limitations remain: The immutable fluorescence of the emitters outshines the phosphorescent patterns by roughly one order of magnitude, allowing readout of the PLTs only after the excitation source is turned off. The programming of these systems is a rather slow process, where lowest reported activation times are still >8 s. Here, a material‐focused approach to PLTs with fast activation times of 120 ± 20 ms and high‐contrast under continuous‐wave illumination is demonstrated, leading to accelerated programming on industry relevant time scales and a simplified readout process both by eye and low cost cameras.

A Novel Motion Blur Resistant vSLAM Framework for Micro/Nano-UAVs

Localization and mapping technologies are of great importance for all varieties of Unmanned Aerial Vehicles (UAVs) to perform their operations. In the near future, it is planned to increase the use of micro/nano-size UAVs. Such vehicles are sometimes expendable platforms, and reuse may not be possible. Compact, mounted and low-cost cameras are preferred in these UAVs due to weight, cost and size limitations. Visual simultaneous localization and mapping (vSLAM) methods are used for providing situational awareness of micro/nano-size UAVs. Fast rotational movements that occur during flight with gimbal-free, mounted cameras cause motion blur. Above a certain level of motion blur, tracking losses exist, which causes vSLAM algorithms not to operate effectively. In this study, a novel vSLAM framework is proposed that prevents the occurrence of tracking losses in micro/nano-UAVs due to the motion blur. In the proposed framework, the blur level of the frames obtained from the platform camera is determined and the frames whose focus measure score is below the threshold are restored by specific motion-deblurring methods. The major reasons of tracking losses have been analyzed with experimental studies, and vSLAM algorithms have been made durable by our studied framework. It has been observed that our framework can prevent tracking losses at 5, 10 and 20 fps processing speeds. vSLAM algorithms continue to normal operations at those processing speeds that have not been succeeded before using standard vSLAM algorithms, which can be considered as a superiority of our study.

Distance Measurement Using Dual Laser Source and Image Processing Techniques

Camera based systems are gaining popularity in automotive driver assistance systems. They are used to detect obstacles, road signs, road markings and other road users. Today, even low cost camera technology enables obtaining a very good image resolution. The proposed system presented in this paper can be used to measure the distance or the displacement of an object, by applying laser optics and trigonometry. The vision-based system was developed using a dual laser source and a CCD camera for recording images which are processed to obtain the distance from an object. The image processing techniques used in this paper consists of two main improved processes which are, the detection of a laser spot using image processing and the calculation of the distance based on laser spot position on the image and the relationship between pixel number and distance. It is shown that the proposed system is able to measure distances in a quick and accurate fashion.

Plant Leaf Detection and Counting in a Greenhouse during Day and Nighttime Using a Raspberry Pi NoIR Camera.

A non-destructive method using machine vision is an effective way to monitor plant growth. However, due to the lighting changes and complicated backgrounds in outdoor environments, this becomes a challenging task. In this paper, a low-cost camera system using an NoIR (no infrared filter) camera and a Raspberry Pi module is employed to detect and count the leaves of Ramie plants in a greenhouse. An infrared camera captures the images of leaves during the day and nighttime for a precise evaluation. The infrared images allow Otsu thresholding to be used for efficient leaf detection. A combination of numbers of thresholds is introduced to increase the detection performance. Two approaches, consisting of static images and image sequence methods are proposed. A watershed algorithm is then employed to separate the leaves of a plant. The experimental results show that the proposed leaf detection using static images achieves high recall, precision, and F1 score of 0.9310, 0.9053, and 0.9167, respectively, with an execution time of 551 ms. The strategy of using sequences of images increases the performances to 0.9619, 0.9505, and 0.9530, respectively, with an execution time of 516.30 ms. The proposed leaf counting achieves a difference in count (DiC) and absolute DiC (ABS_DiC) of 2.02 and 2.23, respectively, with an execution time of 545.41 ms. Moreover, the proposed method is evaluated using the benchmark image datasets, and shows that the foreground–background dice (FBD), DiC, and ABS_DIC are all within the average values of the existing techniques. The results suggest that the proposed system provides a promising method for real-time implementation.

Demonstration of >1Tbit/s WDM OWC with wavelength-transparent beam tracking-and-steering capability

Beam tracking-and-steering is crucial for the operation of high-speed, narrow beam, optical wireless communication (OWC) systems. Using a system based on two sets of low-cost cameras for continuous beam tracking and a set of mirrors for steering, we demonstrate here a high-capacity (>1Tbit/s) ten-channel wavelength-division multiplexed (WDM) OWC system based on discrete multitone transmission. The results, which are achieved over a 3.5-m perpendicular distance and across a lateral coverage up to 1.8 m, constitute to the best of our knowledge, the highest aggregate OWC capacity at this coverage.

An autonomous laser kirigami method with low-cost real-time vision-based surface deformation feedback system

We introduce an autonomous laser kirigami technique, a novel custom manufacturing machine system which functions somewhat similar to a photocopier. This technique is capable of creating functional freeform shell structures using cutting and folding (kirigami) operations on sheet precursors. Conventional laser kirigami techniques are operated manually and rely heavily on precise calibrations. However, it is unrealistic to design and plan out the process (open loop) to realize arbitrary geometric features from a wide variety of materials. In our work, we develop and demonstrate a completely autonomous system, which is composed of a laser system, a 4-axis robotic arm, a real-time vision-based surface deformation monitoring system, and an associated control system. The laser system is based on the Lasersaur, which is a 120-Watt CO2 open-source laser cutter. The robotic arm is employed to precisely adjust the distance between a workpiece and the laser lens so that a focused and defocused laser beam can be used to cut and fold the workpiece respectively. The four-axis robotic arm provides flexibility for expanding the limits of possible shapes, compared to conventional laser machine setups where the workpiece is fixed on rigid holders. The real-time vision-based surface deformation monitoring system is composed of four low-cost cameras, an integrated AI-assisted algorithm, and the sensors (detachable planar markers) mounted on the polymer-based sheet precursors, and allows real-time monitoring of the sheet forming process with a geometric feature estimation error less than 5% and delay time around 100 ms. The developed control system manages the laser power, the laser scanning speed, the motion of the robotic arm based on the designed plan, and the close-loop feedback provided by the vision-based surface deformation monitoring system. This cyber-physical kirigami platform can operate a sequence of cutting and folding processes in order to create kirigami objects. Hence, complicated kirigami design products with various different polygonal structures can be realized by undergoing sequential designed laser cuts and bends (at any folding angles within designed geometric tolerance) using this autonomous kirigami platform.

Infrared thermography analysis with a low-cost camera in plaster sculptures of museum collections

Infrared thermography (IRT) is a non-destructive and non-invasive technique that provides the possibility to investigate the surface of sculptures for the detection of subsurface features and anomalies such as delamination, layer structure, fillings, and defects. IRT has been widely used in buildings and large structures, as well as in works of art such as bronze sculptures and paintings. This article describes the application of active infrared thermography, using a portable low-cost IRT camera, in the examination of plaster sculptures, a material where it has not yet been applied to. In particular, it was used in two plaster sculptures by 19th-century Portuguese artist Soares dos Reis, within a wider project (GEO-SR) aimed at the study of his work. The results indicate that thermography is a suitable technique with a great potential to detect alterations under the surface of plaster, revealing a new look into its manufacturing and conservation.

Image Acquisition Device for Smart-City Access Control Applications Based on Iris Recognition.

In this work, we present an eye-image acquisition device that can be used as an image acquisition front-end application in compact, low-cost, and easy-to-integrate products for smart-city access control applications, based on iris recognition. We present the advantages and disadvantages of iris recognition compared to fingerprint- or face recognition. We also present the main drawbacks of the existing commercial solutions and propose a concept device design for door-mounted access control systems based on iris recognition technology. Our eye-image acquisition device was built around a low-cost camera module. An integrated infrared distance measurement was used for active image focusing. FPGA image processing was used for raw-RGB to grayscale demosaicing and passive image focusing. The integrated visible light illumination meets the IEC62471 photobiological safety standard. According to our results, we present the operation of the distance-measurement subsystem, the operation of the image-focusing subsystem, examples of acquired images of an artificial toy eye under different illumination conditions, and the calculation of illumination exposure hazards. We managed to acquire a sharp image of an artificial toy eye sized 22 mm in diameter from an approximate distance of 10 cm, with 400 pixels over the iris diameter, an average acquisition time of 1 s, and illumination below hazardous exposure levels.

Forward collision warning system for motorcyclist using smart phone sensors based on time-to-collision and trajectory prediction

Purpose The purpose of this paper is to develop a proof-of-concept (POC) Forward Collision Warning (FWC) system for the motorcyclist, which determines a potential clash based on time-to-collision and trajectory of both the detected and ego vehicle (motorcycle). Design/methodology/approach This comes in three approaches. First, time-to-collision value is to be calculated based on low-cost camera video input. Second, the trajectory of the detected vehicle is predicted based on video data in the 2 D pixel coordinate. Third, the trajectory of the ego vehicle is predicted via the lean direction of the motorcycle from a low-cost inertial measurement unit sensor. Findings This encompasses a comprehensive Advanced FWC system which is an amalgamation of the three approaches mentioned above. First, to predict time-to-collision, nested Kalman filter and vehicle detection is used to convert image pixel matrix to relative distance, velocity and time-to-collision data. Next, for trajectory prediction of detected vehicles, a few algorithms were compared, and it was found that long short-term memory performs the best on the data set. The last finding is that to determine the leaning direction of the ego vehicle, it is better to use lean angle measurement compared to riding pattern classification. Originality/value The value of this paper is that it provides a POC FWC system that considers time-to-collision and trajectory of both detected and ego vehicle (motorcycle).

Design of a Novel Remote Monitoring System for Smart Greenhouses Using the Internet of Things and Deep Convolutional Neural Networks

To support farmers and improve the quality of crops production, designing of smart greenhouses is becoming indispensable. In this paper, a novel prototype for remote monitoring of a greenhouse is designed. The prototype allows creating an adequate artificial environment inside the greenhouse (e.g., water irrigation, ventilation, light intensity, and CO2 concentration). Thanks to the Internet of things technique, the parameters controlled (air temperature, relative humidity, capacitive soil moisture, light intensity, and CO2 concentration) were measured and uploaded to a designed webpage using appropriate sensors with a low-cost Wi-Fi module (NodeMCU V3). An Android mobile application was also developed using an A6 GSM module for notifying farmers (e.g., sending a warning message in case of any anomaly) regarding the state of the plants. A low-cost camera was used to collect and send images of the plants via the webpage for possible diseases identification and classification. In this context, a deep learning convolutional neural network was developed and implemented into a Raspberry Pi 4. To supply the prototype, a small-scale photovoltaic system was built. The experimental results showed the feasibility and demonstrated the ability of the prototype to monitor and control the greenhouse remotely, as well as to identify the state of the plants. The designed smart prototype can offer real-time remote measuring and sensing services to farmers.

Automating the Clinical Assessment of Independent Wheelchair Sitting Pivot Transfer Techniques.

Using proper transfer technique can help to reduce forces and prevent secondary injuries. However, current assessment tools rely on the ability to subjectively identify harmful movement patterns. The purpose of the study was to determine the accuracy of using a low-cost markerless motion capture camera and machine learning methods to evaluate the quality of independent wheelchair sitting pivot transfers. We hypothesized that the algorithms would be able to discern proper (low risk) and improper (high risk) wheelchair transfer techniques in accordance with component items on the Transfer Assessment Instrument (TAI). Transfer motions of 91 full-time wheelchair users were recorded and used to develop machine learning classifiers that could be used to discern proper from improper technique. The data were labeled using the TAI item scores. Eleven out of 18 TAI items were evaluated by the classifiers. Motion variables from the Kinect were inputted as the features. Random forests and k-nearest neighbors algorithms were chosen as the classifiers. Eighty percent of the data were used for model training and hyperparameter turning. The validation process was performed using 20% of the data as the test set. The area under the receiver operating characteristic curve of the test set for each item was over 0.79. After adjusting the decision threshold, the precisions of the models were over 0.87, and the model accuracies were over 71%. The results show promise for the objective assessment of the transfer technique using a low cost camera and machine learning classifiers.