Real-time Camera Images Research Articles

Prototype Smart Car Mecanum Wheel Fire Extinguisher Based On Esp 32 Cam With Bluetooth Combination

The development of microcontroller technology has been implemented in the robotics industry, particularly in performing tasks that are risky for living beings, especially humans. In this context, a wheeled smart car robot can automatically extinguish fires. The fire extinguishing system is taken over by the robot's system when fire or heat is detected, and the robot's controller activates the extinguishing system, spraying water at the fire point. The ESP 32 CAM microcontroller serves as the processing and controlling unit, handling data and displaying real-time camera images to ensure accurate fire extinguishing. Additionally, a mecanum wheel drive mechanism is added to enhance the robot's movement flexibility.

Pick-and-place application using a dual arm collaborative robot and an RGB-D camera with YOLOv5

Nowadays, many industries use robots and cameras in tandem to detect specific objects and perform specific tasks. However, misdetection can occur due to inconsistencies in lighting, background, and environment. In order to address the aforementioned issues, this study proposes using a dual arm six-degree-of-freedom (6-DoF) collaborative robot, ABB YuMi, and red, green, blue-depth (RGB-D) camera with YOLOv5 in a pick-and-place application. In order to prepare the dataset, the images are collected and labeled. The dataset has been trained with the YOLOv5 machine learning algorithm. It has taken on the role of weight for real-time detection. When RGB images from a camera are sent to YOLOv5, data pertaining to the bottle’s position x-y and color are extracted from the depth and color images. The position of the robot is used to control its movement. There are three parts to the experiment. To begin, YOLOv5 is tested with and without trained images. Second, YOLOv5 is tested with real-time camera images. Finally, we assume that YOLOv5 has perfect detection and grasping ability. The results were 95, 90, and 90%.

Real-time detection of rice phenology through convolutional neural network using handheld camera images

Smallholder farmers play an important role in the global food supply. As smartphones become increasingly pervasive, they enable smallholder farmers to collect images at very low cost. In this study, an efficient deep convolutional neural network (DCNN) architecture was proposed to detect development stages (DVS) of paddy rice using photographs taken by a handheld camera. The DCNN model was trained with different strategies and compared against the traditional time series Green chromatic coordinate (time-series Gcc) method and the manually extracted feature-combining support vector machine (MF-SVM) method. Furthermore, images taken at different view angles, model training strategies, and interpretations of predictions of the DCNN models were investigated. Optimal results were obtained by the DCNN model trained with the proposed two-step fine-tuning strategy, with a high overall accuracy of 0.913 and low mean absolute error of 0.090. The results indicated that images taken at large view angles contained more valuable information and the performance of the model can be further improved by using images taken at multiple angles. The two-step fine-tuning strategy greatly improved the model robustness against the randomness of view angle. The interpretation results demonstrated that it is possible to extract phenology-related features from images. This study provides a phenology detection approach to utilize handheld camera images in real time and some important insights into the use of deep learning in real world scenarios.

Field-Programmable System-on-Chip-Based Control System for Real-Time Distortion Correction in Optical Imaging

The digital transition requires real-time control of complex systems with short loop time and low latency in various applications. Field-programmable gate arrays (FPGAs) are, in principle, capable of complying with this task but demand, on the other hand, a high programming effort. In this article, we propose a field-programmable system on chip (FPSoC) as a hybrid solution of an FPGA and a central processing unit (CPU) on a single monolithic die to combine the strengths of both architectures. An FPSoC-based adaptive optical wavefront correction system is presented as a case study to correct camera images in real time that are distorted by time-varying aberrations. While a short total loop time is achieved by interfacing the camera and a deformable mirror on a low level directly with the FPGA, all computationally nonintensive tasks are implemented on the CPU to keep the flexibility, reusability, and development expense low. The system corrects the optical distortion of water surface waves with up to 3600 control cycles per second and spatially attenuates the distortion up to Zernike polynomial 14 with up to 150 Hz. The FPSoC system enables fast spatiotemporal aberration correction in technical processes and offers a perspective for measuring complex flows through fluctuating interfaces.

Open source modules for tracking animal behavior and closed-loop stimulation based on Open Ephys and Bonsai

Objective. A major goal in systems neuroscience is to determine the causal relationship between neural activity and behavior. To this end, methods that combine monitoring neural activity, behavioral tracking, and targeted manipulation of neurons in closed-loop are powerful tools. However, commercial systems that allow these types of experiments are usually expensive and rely on non-standardized data formats and proprietary software which may hinder user-modifications for specific needs. In order to promote reproducibility and data-sharing in science, transparent software and standardized data formats are an advantage. Here, we present an open source, low-cost, adaptable, and easy to set-up system for combined behavioral tracking, electrophysiology, and closed-loop stimulation. Approach. Based on the Open Ephys system (www.open-ephys.org) we developed multiple modules to include real-time tracking and behavior-based closed-loop stimulation. We describe the equipment and provide a step-by-step guide to set up the system. Combining the open source software Bonsai (bonsai-rx.org) for analyzing camera images in real time with the newly developed modules in Open Ephys, we acquire position information, visualize tracking, and perform tracking-based closed-loop stimulation experiments. To analyze the acquired data we provide an open source file reading package in Python. Main results. The system robustly visualizes real-time tracking and reliably recovers tracking information recorded from a range of sampling frequencies (30–1000 Hz). We combined electrophysiology with the newly-developed tracking modules in Open Ephys to record place cell and grid cell activity in the hippocampus and in the medial entorhinal cortex, respectively. Moreover, we present a case in which we used the system for closed-loop optogenetic stimulation of entorhinal grid cells. Significance. Expanding the Open Ephys system to include animal tracking and behavior-based closed-loop stimulation extends the availability of high-quality, low-cost experimental setup within standardized data formats serving the neuroscience community.

Characterization of mountain drainage patterns for GPS-denied UAS navigation augmentation

We present a novel approach to use mountain drainage patterns for GPS-denied navigation of small unmanned aerial systems, such as the ScanEagle, utilizing a down-looking fixed focus monocular imager. Our proposal allows for extension of GPS-denied missions in mountain areas, with no assumption of human-made geographic objects. We leverage the analogy between mountain drainage patterns, human aeteriograms, and human fingerprints, to match local drainage patterns to parallax occlusion maps of geo-registered radar returns (GRRR). The GRRR data may be loaded on-board the aircraft pre-mission to avoid the need for a scanning aperture radar during the mission. For recognition purpose, we represent a given mountain area with a set of spatially distributed mountain minutiae, i.e., details found in the drainage patterns, so that conventional minutiae-based fingerprint matching approaches can be used to match real-time camera images against template images in the training set. We use medical arteriography processing technique to extract the patterns. The minutiae-based representation of mountains is achieved by first exposing mountain ridges and valleys with a series of filters and then extracting mountain minutiae from these ridges/valleys. Our results are experimentally validated on actual terrain data and show the effectiveness of minutiae-based mountain representation method.

Tracking Method of Adaptive For Inserting an Artificial Lens

We marked with a pen directly to the incision for insertion into the patient's eye of the intraocular lens insertion in cataract surgery, and insertion point. However, it can delete or blur the display due to such factors as foreign material coming out of the eye during surgery. Therefore, in recent years, a method of tracking the image processing by receiving the camera images in real time display method such extended. There are many ways to estimate the object in the video in, but in the method has been applied widely used typically have optical flow and template matching. The template matching, while indicating the tracking performance with excellent background simple from complex background, its performance is reduced, while indicating the tracking excellent performance in a complex background, simple background, optical flow, the performance fall. In this paper, there is provided a method adaptive tracking don't give a significant effect on the change in the background template region. Therefore, by using to suit the circumstances the optical flow indicating the tracking excellent performance in a complex background template matching showing a tracking superior performance with a simple background, when inserting an intraocular lens in cataract surgery, and a knife in it was possible to improve the accuracy of tracking a region of the incision.

템플릿 매칭과 버퍼를 이용한 특징점 추적 방법

Today, a surgery for cataract of Korean society frequently come to operation. A method for treating cataracts have been developed in various ways. Widely used method is a method to use the artificial lens and replace it with the existing lens. The surgery can be inserted exactly according to the angle and points that are calculated in advance in the intraocular lens insertion is important. However, The lens insertion point can delete or blur the display due to such factors as foreign material coming out of the eye during surgery. Therefore, The lens insertion point needs a method of tracking the image processing by receiving the camera images in real time display method. In this paper, we propose a feature point tracking method by using template matching and buffer. a simulation results show that our ideas can track a feature point of the intraocular lens insertion.

Augmented Reality-based Support System for Teaching Hand-drawn Mechanical Drawing

This research aims to reduce the time, effort and monetary cost needed for developing various educational materials for teaching hand-drawn mechanical drawing by replacing actual educational materials with virtual ones on an augmented reality (AR) -based support system. This system is used for training of sketching and orthographic projection drawing, and has following features. 1) The virtual educational materials constructed by using 3D-CAD are superimposed to a real-time camera image as AR objects. 2) Students can watch the objects from various directions by using a cubic-marker intuitively. 3) The objects are applied realistic textures similar to the actual materials.

FPGA and ASIC implementation of the algorithm for traffic monitoring in urban areas

FPGA and ASIC implementation of the algorithm for traffic monitoring in urban areas This paper describes the idea and the implementation of the image detection algorithm, that can be used in integrated sensor networks for environment and traffic monitoring in urban areas. The algorithm is dedicated to the extraction of moving vehicles from real-time camera images for the evaluation of traffic parameters, such as the number of vehicles, their direction of movement and their approximate speed. The authors, apart from the careful selection of particular steps of the algorithm towards hardware implementation, also proposed novel improvements, resulting in increasing the robustness and the efficiency. A single, stationary, monochrome camera is used, simple shadow and highlight elimination is performed. The occlusions are not taken into account, due to placing the camera at a location high above the road. The algorithm is designed and implemented in pipelined hardware, therefore high frame-rate efficiency has been achieved. The algorithm has been implemented and tested in FPGA and ASIC.

Low-complexity camera digital signal imaging for video document projection system

We present high-performance and low-complexity algorithms for real-time camera imaging applications. The main functions of the proposed camera digital signal processing (DSP) involve color interpolation, white balance, adaptive binary processing, auto gain control, and edge and color enhancement for video projection systems. A series of simulations demonstrate that the proposed method can achieve good image quality while keeping computation cost and memory requirements low. On the basis of the proposed algorithms, the cost-effective hardware core is developed using Verilog HDL. The prototype chip has been verified with one low-cost programmable device. The real-time camera system can achieve 1270 × 792 resolution with the combination of extra components and can demonstrate each DSP function.

FPGA-Based Real-Time Implementation of Detection Algorithm for Automatic Traffic Surveillance Sensor Network

This paper describes the FPGA-based hardware implementation of an algorithm for an automatic traffic surveillance sensor network. The aim of the algorithm is to extract moving vehicles from real-time camera images for the evaluation of traffic parameters, such as the number of vehicles, their direction of movement and their approximate speed, using low power hardware of a sensor network node. A single, stationary, monochrome camera is used, mounted at a location high above the road. Occlusions are not detected, however simple shadow and highlight elimination is performed. The algorithm is designed for frame-rate efficiency and is specially suited for pipelined hardware implementation. The authors, apart from the careful selection of particular steps of the algorithm and the modifications towards parallel implementation, also proposed novel improvements such as backgrounds' binary mask combination or non-linear functions in highlight detection, resulting in increasing the robustness and efficiency of hardware realization. The algorithm has been implemented in FPGA and tested on real-time video streams from an outdoor camera.

A Novel Vision-based Location Authentication Approach in a Ubiquitous Camera Environment

Location-based services (LBSs) have been emerging in recent years. Mobile devices with built-in GPS receiver (Global Positioning System), called GPS-enabled mobile devices, will also become an important trend in the near future. Location information and authentication of mobile clients are critical for accessing desired LBSs. In this paper, a novel vision-based approach, called VLocAuth, is proposed to incorporate a ubiquitous camera (UbiCam) environment for location authentication. The operation of VLocAuth relies on three kinds of servers, including LBS server, authentication server (AS), and camera server (CS). VLocAuth consists of two phases: initialization and location authentication phases. In the initialization phase, a mobile client requests user identification and related information from LBS server via a secure channel. In the location authentication phase, a temporal identification provided by LBS server is used for data communication among the client, AS, and CS to achieve privacy protection. Then, CS authenticates the location of a mobile client by matching the location and the moving objects in the real-time camera image to ensure the client at the location of the GPS coordinates. The security analysis shows that VLocAuth is secure against replay attack and man-in-the-middle attack. It also satisfies unforgeability, privacy, confidentially, integrity, simplicity, and practicability requirements. A simulation study is designed to show the influence of a node density, type of GPS receiver, and a network delay on VLocAuth. Besides, a matching tool is implemented for measuring the time of the key matching step of VLocAuth in a practical environment. These results show that VLocAuth is feasible for location authentication and meets the trend of UbiCam environment in the near future.

Automatic Camera Calibration and Rectification Methods

The Global Positioning System (GPS) system is popularly used in an outdoor environment. A GPS tracking system usually displays the location of a moving object on a digital map. It is difficult to realize its actual situation without visual information. Therefore, cameras are incorporated with a GPS in order to provide a new GPS-based visual tracking service (GPS-VT). The tracking of an object with a GPS receiver can be performed not only on the digital map but also on the real-time camera image. GPS-VT was proposed in our previous study. It relies on the camera calibration for the transformation from a GPS coordinate to the coordinate on the camera image. Five point correspondences, i.e., a GPS coordinate and its corresponding image coordinate, must be established to generate the intrinsic and extrinsic parameters of the camera needed for coordinate transformation. There are two drawbacks in calibrating a camera. One is the difficulty in establishing five point correspondences manually for an uncalibrated camera. The other is the inconvenience of recalibrating a calibrated camera after adjusting its field-of-view (FOV). In this paper, an automatic calibration method and a rectification method are proposed to overcome separately the above-mentioned drawbacks. For an uncalibrated camera, the GPS moving vectors of the operator are matched with those moving vectors of objects in the camera image to locate the operator. The operator simply walks to five different locations and the point correspondences can be established automatically. For the rectification of a calibrated camera, two sets of feature points are extracted separately from the images before and after the adjustment of the FOV. The correlations of feature points of two images are analyzed by using a normal cross correlation (NCC) method to obtain the offsets of the adjustment in x and y directions. The offsets are used to rectify the image coordinate transformed from the GPS coordinate. Two experiments were also designed for both methods separately. For an uncalibrated camera, the locating errors are estimated and compared for the five point correspondences that are established manually or automatically. The difference in locating errors of manual and automatic calibration only totalled 2.6 pixels. The results show that the accuracy of the automatic calibration method is almost the same as that of the manual calibration. For the rectification of a calibrated camera, the differences in locating errors are estimated for various adjustment offsets in x and y directions. The errors of the proposed method totalled less than five pixels for the adjustment up to 200 pixels, without recalibrating the camera. The tests show that the result of the rectification method is close to that of manual calibration. Therefore, the two proposed methods overcome the problems of using the GPS-VT in an actual environment.

A new software/hardware architecture for real time image processing of wide area airborne camera images

This paper describes a new software/hardware architecture for processing wide area airborne camera images in real time. The images under consideration are acquired from the 3K-camera system developed at DLR (German Aerospace Center). It consists of three off-the-shelf cameras, each of it delivers 16 Mpixel three times a second. One camera is installed in nadir, whereas the other two cameras are looking in side direction. Main applications of our system are supposed to be automotive traffic monitoring, determining the workload of public road networks during mass events, or obtaining a survey of damages in disaster areas in real time. Altogether, this demands a fast image processing system on the aircraft, because the amount of original high resolution images can not be sent to ground by up-to-date transfer mode systems. The on-board image processing system is distributed over a local network. On each PC several modules are running concurrently. In order to synchronize several processes and to assure access to commonly used data, a new distributed middleware for real time image processing is introduced. Two sophisticated modules one for orthorectification of images and one for traffic monitoring are explained in more detail. The orthorectification and mosaicking is executed on the fast graphics processing unit on one PC, whereas the traffic monitoring module runs on another PC in the on-board network. The resulting image data and evaluated traffic parameters are sent to a ground station in near real time and are distributed to the involved users. Thus, with the here suggested software/hardware system it becomes possible to support rescue forces and security forces in disaster areas or during mass events in near real time.

Implementation of a fast Fourier transform (FFT) for image processing applications

Different fast Fourier transform (FFT) algorithms for hardware implementation have been considered. We propose an implementation whereby two radix-N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sup> passes are carried out in parallel and in which each N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sup> -point transform is carried out via a serial input parallel output transform circuit. The processing rate is one clock cycle per input point for the N-point transform regardless of the value of N chosen. The circuit is being implemented with TTL logic and will be used to perform spatial frequency domain filtering on two dimensional infrared camera images in real time; real time meaning processing between frame display.